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Application development inspections is a disciplined technical 
project review technique used at the end of various application 
development phases to improve program quality and project 
manageability and to increase productivity. This manual is 
designed to provide enough information to the manager of the 
data processing activity and his technical staff to answer the 
question: “Should this activity use inspections?” The manual 
also provides guidance in preparing for and performing 
inspections. A case study is included. 


This description of the inspections technique is made 
available by IBM with the objective of providing information 
that may assist others in improving their own application 
development procedures. It does not require the use of IBM 
products or services. However, IBM DP Services will, subject 
to the local availability of appropriate resources, respond to 
requests for assistance in implementing inspections. 
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Preface 


Application development inspections is a technical project review technique held 
at the end of various application development phases. 


This manual is designed to provide enough information to the manager of the data 
processing activity and his technical staff to answer the question: “Should this 
activity use inspections?” The manual also provides guidance for the moderator — 
the person who schedules inspections, invites participants, and controls inspection 
meetings and their follow-up — in preparing for and performing inspections. 
Appendixes provide a case study and solution that can be used in moderator 
training, offer sample materials that can be modified for use during inspections, 
and describe two types of inspections, test plan and test case, that installations 
may wish to adopt after gaining experience with the other types of inspections 
described in the text. 


Although the inspections technique was developed within IBM in a system control 
programming environment one appendix reports on the use of inspections in an 
application development project within IBM. Each installation must evaluate the 
technique’s usefulness for its own environment and, if adopted, tailor such 
elements as exit and reinspection criteria, checklists, and time estimates to fit the 
installation’s characteristics. 


The inspections technique is also discussed in Design and Code Inspections to 
Reduce Errors in Program Development, an article by M. E. Fagan that appeared 
in the IBM Systems Journal, Volume 15, Number 3, 1976. A reprint of this 
article is available from IBM under order number G321-5033. 


First Edition (July 1977) 


Requests for copies of IBM publications should be made to your IBM representative or to 
the IBM branch office serving your locality. 


Address comments concerning the contents of this publication to IBM Corporation, 
Technical Publications, Dept. 824, 1133 Westchester Avenue, White Plains, N.Y. 10604. 


© Copyright International Business Machines Corporation 1977 
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Chapter 1. Introduction 


Project reviews at the end of various application 
development phases have long been recognized as a 
vehicle for determining application development 
project status and identifying areas that need special 
attention. Such reviews are usually intended to 
accomplish the following: 

1. Communicate project status to higher-level 
management and/or project personnel. 

2. Establish that the project can be completed 
within the time and costs allotted, or adjust such 
schedules and costs. 

3. Evaluate the technical accuracy of the project 
and arrange for the correction of technical 

_ errors uncovered as early as possible in the 
development cycle. 

In practice, however, project reviews tend to omit 

this last function. 


The Need for Technical Reviews 

Technical reviews during which technical accuracy is 
evaluated and arrangements for necessary correc- 
tions made are needed at various points within the 
development cycle to help produce a product of 
higher quality that is easier to maintain while reduc- 
ing the cost of finding and correcting errors and to 
help management to better control the development 
process (see Figure 1). 

Higher quality and improved maintainability are 
central objectives of data processing management. 
The number of completed programs in use has been 
increasing; many of them still contain errors and 
_ frequently are difficult to modify; as a result, the 
number of personnel devoted to program mainte- 
nance has also been steadily increasing. It has been 
estimated that 40-60 percent of every 


Project status review 


Management 


Management | 
Developers 


Technical reviews 


Figure 1. Technical reviews versus project status reviews 


Type of Review Function/Objectives 


® Communicate project status 
@ Check and adjust cost and schedules 


Improve quality 
Improve maintainability 
Technical evaluation 

Reduce costs 

Improve management control 


programmer/analyst staff dollar is devoted to the 
maintenance of operational programs. As the size 
and complexity of new programs continue to rise, 
this is likely to increase further. 

Achieving quality and maintainability goals while 
reducing the cost of finding and correcting errors 
dictates a strategy of early error detection. The cost 
of finding and correcting errors increases substan- 
tially as the development process continues. That is, 
the cost of detection and correction of errors during 
testing is dramatically higher then during design; the 
expense is higher still after the program has been 
placed into productive use. 

Technical evaluation of product quality at the end 
of development phases can help improve manage- 
ment control by providing more realistic evaluations 
of the developing product’s quality, cost, and sched- 
ule status, and can make possible timelier corrective 
action. 


Structured Walk-Throughs and 
Inspections 

Two types of technical reviews now in use are 
structured walk-throughs and application develop- 
ment inspections. The fundamental differences 
between the two are discussed in this text; the major 
concern, however, is inspections. 


Structured Walk-Throughs 

In a structured walk-through, a developer’s work 
(program design, code, documentation, etc.) is 
reviewed by fellow project members invited by the 
developer. Structured walk-throughs, in various 
forms, are being used by many project development 
groups. The forms vary in their objectives and 


procedures, but most have the following 
characteristics: 


i 


2 


They are arranged and scheduled by the devel- 
oper of the work product being reviewed. 
Management does not ordinarily attend the 
walk-through. 


. The developer selects the list of reviewers, but 


in most cases, management examines the list to 

ensure that developers of related products are 

invited. The walk-through is usually attended 

by four to six reviewers. Participants can 

include: 

¢ Designers of the system, to ensure compatibil- 
ity and continuity of design | 

e Individuals responsible for documenting the 
function being reviewed 

« Testers responsible for functional and system 
testing , 

e Developers of other parts of the system 

e Developers of other systems that interface 
with the one being reviewed 


. The reviewers are given the materials.four to six 


days before the walk-through and are expected 
to review them and come to the session with a 
list of questions. 


. A typical walk-through is scheduled to last for a 


specified period, not longer than two hours. If 
the materials have not been completely re- 
viewed at the end of that period, or if a signifi- 
cantly large list of issues have been created, 
another walk-through is scheduled for the next 
convenient time. 


. One person — usually the author whose work is 


being examined, or perhaps a project team 
leader — is appointed or elected to guide the 
session. That person compiles an action list 
consisting of all errors, discrepancies, exposures, 
and inconsistencies uncovered during the walk- 
through. 


. All issues are resolved after the session. The 


walk-through provides problem detection, not 
problem resolution. 


Inspections | 

Inspections provide a more formal and rigorous 
method of performing technical reviews at the end 
of development phases. Application development 
inspections, as used in IBM development groups, can 
be characterized as follows: 


i 


Inspections appear as separate scheduled © 
activities in the project development plan, and 


the project schedule contains a time allowance 


for rework of deficiencies identified in the 
inspection process. 


2. Each development phase, at the end of which 


work products are to be inspected, is defined, as | 
are the exit criteria for each. 


. At the end of an inspection, formal approval is 


required that deficiencies have been satisfactori- | 
ly reworked and exit criteria have been satisfied 
before work can proceed to the next develop- 
ment phase. 


. A specially trained moderator schedules and 


conducts the inspections and chooses the partic- 
ipants, who are usually selected on the basis of 
special skills or knowledge. The moderator is 
not the developer of the product being inspect- 
ed nor a member of the development team; he 
does not usually devote full time to this role, but 
is a working analyst or programmer. A 
moderator’s work may be inspected in turn by 
inspectors from other projects. 


. The inspection of a work product at the end of 


a development phase consists of six well- 
defined steps, as shown in Figure 2. 


.. The inspections technique emphasizes the 


accumulation and analysis of data about the 
types of errors and their frequency of occur- 
rence. As the data base of error patterns grows 
larger and the moderators gain experience with 
error detection, the moderators can better 
analyze the results of inspections, can better 
help designers and implementers avoid errors, 
and can help inspection teams learn to doa 
more thorough job of error detection. Check- 
lists developed from the data base assist in this 
by assuring that all reasonable questions have 
been considered during an inspection. Also, by 
comparing current inspection results against the 
data base, both developers and management can 
become aware of situations in which corrective 
action must be taken to avoid schedule delays or 
to reduce the liklihood of creating excessively 
error-prone modules. Accumulation and analy- 
sis of error patterns can also highlight for 
management those development practices in 
need of revision or suggest some that could be 
initiated, thus leading to an improved develop- 
ment process. 

In an installation that requires several modera- 
tors, management chooses one to serve as an 
inspection coordinator to oversee the implemen- 
tation of inspections for the installation. The 
coordinator controls the establishment and 
modification of the inspection procedures. He 
normally serves as the first moderator; trains 


_ subsequent moderators; standardizes record- 


keeping procedures, checklists, and exit and 


se Inspection Team Participants Objectives 


. Planning Moderator Scheduling/ distributing’ 
materials 


. Overview Designer and other participants Education 


. Preparation Participants — individually | Self-study 


. Inspection Entire group Finding errors 
meeting 


. Rework Author Correcting errors 


. Follow-up Moderator, author. Assuring correct rework, 
improving development 
and inspections 


Figure 2. The six steps of an inspection (summarized) 


reinspection criteria; maintains the installation’s as is the case with inspections, although a second 
inspections data base; and, by analyzing this walk-through is usually performed when a “‘large”’ 
information, provides such data as time esti- number of errors has been detected. In the 
mates for the various inspection steps, expected | inspections discipline, “‘exit criteria’ for each 
error rates, and analyses of errors by type. The development phase must be satisfied, errors 
coordinator’s analyses of the data also assist the corrected, and formal approval given before the 
moderators in maintaining a high level of effec- work can proceed to the next phase. 
tiveness and efficiency during inspections. «The moderator of the inspection is not part of the 
7. An application development process that team that developed the work product. He is 
includes inspections can be compared to an trained in the skills required for the moderator’s 
industrial continuous flow process in that each role: planning the inspections, selecting partici- 
provides the environment for better manage- pants, preparing for the inspection meeting, 
ment control of the process because manage- maintaining an efficient pace during the inspec- 
ment (1) defines the operations in the process, tion meeting, assuring that all reasonable error 
(2) defines the criteria for completion of each possibilities have been considered, keeping inter- 
operation, and (3) measures the process by personal friction to a constructive level, recording 
collecting data about the functioning of the and categorizing errors, following up on rework, 
process. and analyzing inspection results. He carries the 
experience he gains forward from project to 
Key Differences Between Inspections and Structured project, and becomes an expert in making the 
Walk-Throughs | most effective possible use of the participants’ 
Key differences exist although there are variations of time. 
both techniques. Because of the variations, one or Procedural differences include the following: 
more of these differences may not be applicable in 1. The inspection procedure is divided into six 
some installations. distinct steps, each of which has its own stated 
The differences may be characterized as follows: objectives. In walk-throughs, these steps exist 
¢ Typically, walk-throughs are characterized by but are blended together, with several objec- 
informality, with the developer (or chief program- tives being addressed simultaneously. 
mer) requesting them. They may be performed 2. At a walk-through meeting, the developer of the 
on completed activities or during development of work product usually conducts the meeting and 
an activity. Although proper follow-up usually “reads” the materials. In inspections, the 
ensues, no formal approval is generally required moderator conducts the meeting and designates 


before proceeding with further development work — someone other than the developer to read the 


materials so that the developer’s interpretation 
cannot inadvertently cover up an error. 

3. Errors detected during the preparation period 
and discussed with the developer are usually not 
brought up at a walk-through meeting. During 
inspection meetings, a// errors are noted and 
described to establish error patterns, improve 
the skills of all the participants, and flag any 
schedule slippage as early as possible. 

4. Management does not usually participate in 
meetings in either form of review. In the case 
of structured walk-throughs, they do not attend 
because their presence may interfere with the 
free flow of discussion among the working 
group. In the case of inspections, managers are 
not discouraged from attending, but usually 
cannot add valuable input to an essentially 
technical discussion; they are therefore not 
needed. Management is, however, informed of 
the results of inspection meetings. In addition 
to traditional quantitative data, such as lines of 
code completed, management receives data 
regarding program quality (error rates) and time 
expected for rework and retesting. 

5. Inspections emphasize uniformity and complete- 
ness through the use of checklists. 

6. Errors are categorized during an inspection 
meeting and entered into a data base to improve 
the developers’ and management’s understand- 
ing of the types of errors that are occurring and 
where they most often occur. This information 
can be used to develop acceptable error rate 
standards to determine whether a module 
should be reinspected or rewritten. It can also 
assist management, testers, and those who are 
developing related modules to plan their work 
better and maintain more accurate schedules. 

Briefly stated, the walk-through procedure relies 

heavily on technical team self-control and confines 
the visibility of shortcomings to within the develop- 
ment team itself. On the other hand, the inspection 
process allows for such visibility to extend beyond 
the development team and imposes technical con- 
trols from sources (the moderator and management) 
that are external to the team. 

Note that the differences between structured 


walk-throughs and inspections tend to diminish as 
more of the formalities of inspections are included in 
walk-throughs. For instance, walk-throughs are now 
frequently scheduled into a project development 
plan. 


A Controlled Experiment 

Inspections are designed to improve both program 
quality and development productivity. This concept . 
was tested in an experiment conducted by an IBM 
programming department. The inspection sample 
was considered to be representative in terms of its 
size and other characteristics. It was designed and 
developed by three designers and 13 programmers, 
was Structured, and judged to be of moderate 
complexity. In the experiment, inspections were 
held, as shown in Figure 3, after detailed design and 
coding. After each inspection, the necessary rework 
was performed, and the work proceeded to the next 
phase. After including the time to prepare for and 
perform the inspections and do the rework, a net 
saving resulted when compared to similar projects 
using walk-throughs, the technique then in use. It 
was estimated that 94 programmers hours were 
saved per thousand lines of code as a result of the 
detailed design inspection, and 51 hours by the code 
inspection. This net saving translates into a 23 
percent increase in coding productivity. (The unit 
test inspection did not result in a saving of program- 
mer hours and therefore was eliminated subsequent 
to the experiment. | 

To determine that the productivity gain was not a 
result of the participants’ knowledge that they were 
being studied and therefore producing at above 
normal rates, a control sample inspection was 
conducted later, after inspections had become 
accepted as normal practice. Inspection results for 
the control sample were essentially the same as those 
for the original experiment. 

The study also compared the quality of the pro- 
grams produced during the inspection sample study 
with a comparable component produced similarly, | 
but using walk-throughs instead of inspections. | 
Equivalent testing between post unit testing and 
system test showed the inspection sample to contain 
38 percent fewer errors than the walk-through sample. 


Detailed 
design 


Net Coding Productivity 


@® Sample showed 23 percent net increase 


@ Poststudy sample showed 22 percent net increase 


Net Savings in Programmer Hours/1000 LOC 


@ Design inspection: 94 
@ Code inspection: 51 
@ Unittest: -20 


Program Quality 


The inspection sample had 38 percent fewer errors/1000 LOC. 


Figure 3. An IBM sample inspection study 


Implementing Inspections 

The first step in implementing inspections is a 
management decision that the following objectives 
and methods of the inspections technique will be 
appropriate for the installation. 

The principal objectives of inspections are (1) to 
improve application system quality while reducing 
the cost of application development and mainte- 
nance and (2) to improve management’s ability to 
control the development process. These objectives 
are accomplished by (1) providing a uniform method 
for inspecting development materials, (2) verifying 
that development materials are complete (satisfy the 
exit criteria for that development phase), (3) detect- 
ing errors and ambiguities in design and code at the 
earliest possible point in the development before 
they become progressively more expensive to 
rework, (4) verifying that all errors discovered 
during an inspection have been corrected before the 
work product (for example, a coded module) being 
inspected moves to the next development phase, 

(5) maintaining and analyzing records of errors 
found and time spent in inspections to improve both 
the development process and the inspections tech- 
nique, and (6) providing early clues to the quality of 
developing products. | 

Following a management decision to implement 
inspections, full management support of the tech- 


nique, as well as awareness by the development staff 


of management’s support, is necessary for successful 


implementation. 

The next step is to choose a pilot project. It is 
recommended that the project be currently in 
development and proceeding without any unusual 
difficulty. For best results, it should not be under 
any abnormal time constraints so that the technique 
can be judged in as neutral an environment as 
possible. It is advisable to choose a project for 
which all three major types of inspections (initial 
design, detailed design, and coding) can be used for 
each module to be inspected. No more than three or 
four months should elapse from the time that inspec- 
tions are first used to the time that testing starts so 


that the developers can evaluate the results within a 
reasonable time period. 


The moderator chosen for the pilot project, as for 


any inspection, should not be a member of the team 


whose work product is to be inspected because the 
moderator needs to bring an objective, outside 
viewpoint to the inspections. For the pilot project, 
however, it is desirable for the moderator to report 
to the same manager as that of the pilot project so 
that both the costs and the benefits of the inspec- 
tions process are under the same control. The 
moderator will probably need to establish exit and 
reinspection criteria based on installation require- 


ments and standards. There may also be a need to 


modify the reporting forms and checklists included 
in this text. 

Beginning with the first inspection of the pilot 
project, the moderator should conscientiously 
maintain the records because the accumulation and 
analysis of inspection and man-hours data provide 
some of the major benefits of inspections. For 
example, the analyses can help to answer such 
questions as: 

e How likely are we to meet our scheduled dates? 

e How likely is this program or module to experi- 

ence excessive maintenance? 

e Are we spending too much or too little time on 

inspections? 

e When should we reinspect a module? 

e When should we rewrite a module? | 

As with any new technique, inspections will not 
experience immediate acceptance. Initially, there 
will be misgivings among the participants, and a 
certain amount of defensiveness can be expected 
from those whose work is inspected first, but these 
reactions should disappear as soon as the developers 
realize that they are not being singled out for criti- 


‘cism. The usual experience has been that the 


developers overcome their initial hesitation, begin to 
count on the constructive feedback they receive 
from their peers, and gain new confidence in their 
ability to predict project completion times. Once the 
pilot project has proven itself, the participants 
usually have a positive attitude toward inspections 
and should, by informal contact with other develop- 
ment personnel, help to generate interest in the . 
technique. They can also be trained as moderators 
and can introduce additional groups or individuals to 
inspections. 

Experience indicates that success with inspection 

pilot projects is usually associated with: 

e Success of the inspections themselves (successful 
inspections find a significant number of serious 
errors) 

e Conduct of the moderator (the moderator should 
encourage a constructive environment and use the 
participants’ time effectively) 

e Attitude of management (management must 
recognize the potential benefits of the inspection 
process, support the pilot effort, and use the 
results to improve the development process) 


Chapter 2. The Inspection Technique 


Inspections are described here as they have been 
used in a system control programming environment 
in IBM, although some terminology changes have 
been made to make the document more meaningful 
to an application development environment. 
Changes to the sample exit and reinspection criteria, 
checklists, time estimates, and reporting forms will 
be necessary to adapt the technique to each 
installation’s environment. 


Inspection Types 

Inspections have been most commonly introduced at 
three “inspection points’’: immediately following 
initial design, detailed design, and coding. They are 
also used extensively within IBM following the 
development of test plans and test cases (see 

Figure 4). — 

Inspections can also be given an important role in 
the review of publications and requirements. The 
major part of this text discusses initial design, 
detailed design, and code inspections. Test plan and 
test case inspections are discussed in Appendix F. 

Note that inspections can be used equally well in 
conventional and top-down program development 
environments. In the latter case, each module or 
group of modules in an application passes through 
the same development phases, although one module 
may do so weeks or months ahead of another. 


Exit Criteria 

One of the most important checks made in an 
inspection is to determine whether the work product 
has met the exit criteria for that phase and is thus 
eligible to proceed to the next development phase. 
Clearly defined exit criteria are necessary for this 
determination and for consistent error and man- 
hours inspection data. Appendix C contains sample 
exit criteria developed within IBM in a system control 
programming environment for the initial design, 
detailed design, and coding phases; some examples 
from this Appendix follow: 

e For the initial design phase, each design process 
statement (HIPO statement, flowchart block, 
pseudocode statement) should be at a level of 
detail equivalent to 15-25 lines of executable 
source code. | 

¢ For the detailed design phase, each design process 
statement should be equivalent to 3-10 lines of 
executable source code. | 

e For the coding phase, the first clean compilation 
listing should be available. 


Initial 


design 


Initial 
design 
inspection 


Detailed 
design 


Test plan 
preparation 


Detailed 
design 
inspection 


Code 
inspection 


Unit 

test 
Function 
test 


Test 
plan 
inspection 


Test case 
preparation. 


Test case 
inspection 


| System 
test 


Figure 4. Inspection types 


Steps 
An inspection consists of six steps (see Figure 5): 

1. Planning, during which the moderator schedules 
inspection activities and makes sure that inspec- 
tion materials have been distributed 

2. Overview, presented by the author of the 
materials to those who are to participate in the 
inspections 

3. Preparation, during which the participants study 
the materials 

4. Inspection meeting itself, during which the 
participants concentrate on finding errors in the 
materials 

5. Rework, wherein the author corrects the errors 
found in the meeting and summarized and 
reported by the moderator 

6. Follow-up (this step usually involves only the 
author, the moderator, and in larger installa- - 
tions, the inspection coordinator), during which 
the moderator certifies the author’s rework and 
authorizes the next development phase, and 
during which inspection data is analyzed 

Each step of the process has its own objectives. 

The moderator’s objective in the planning step is to 
schedule the overview and inspection meetings and 
to ensure that all the materials are distributed to the 
participants. The purpose of the overview is to 
teach the participants the principles of the design or 
code to be inspected. Participants use the prepara- 
tion step to become thoroughly familiar with the 
inspection materials. The moderator assures himself 
that the participants are adequately prepared before 
holding the inspection meeting. The only objective 


. Planning Moderator 


. Overview Designer and other participants 
. Preparation Participants — individually 


. Inspection Entire group 


meeting 


. Rework Moderator, author 


. Follow-up Moderator, author: 


Figure 5. The six steps of an inspection 
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Inspection Team Participants 


of the inspection meeting is to find errors. No 
education normally takes place at this session. In the 
rework step the author corrects errors found in the 
meeting and summarized and reported by the 
moderator. The error summary also keeps manage- 
ment informed about the quality of the developing 
product. The follow-up step has two objectives: to 
validate the correctness of the work done in the 
rework step (the moderator may call on others for 
assistance with this) and to make improvements in 
the application development process and the inspec- 
tion process, through the analysis of inspection data. 
In larger installations, an inspection coordinator 
helps to perform this analysis function. 

_ The cycle may be partially repeated by requiring a 
reinspection, as shown in Figure 6. At the end of 
the inspection meeting, the moderator determines 
whether the errors found were numerous enough to 
warrant a reinspection after rework has been com- 
pleted. 


Participants 

An inspection team is composed of a moderator, the 
author or developer (designer, coder), and one or 
more inspectors. The moderator chooses the inspec- 
tors on the basis of special skills or knowledge they 
can bring to the inspection. The team normally 
consists of four persons. Large teams tend to 
expend too much manpower, while small teams do 
not generate the kind of human interaction that is 
effective in detecting errors. Under certain circum- 
stances, however, a large team may be warranted. 
For example, if there are many linkages to other 


Scheduling/ distributing 


‘materials 


Education 
Self-study 


Finding errors 


Correcting errors (after their 
summarization and reporting) 


Assuring correct rework, 


improving development 
and inspections 


Phase X 


Preparation 


Inspection 
meeting 


Reinspection 
necessary ? 


Next 
phase 


Figure 6. Reinspection points 


modules, designers and/or coders of those other 
modules can contribute to the effectiveness of the 
inspection. 

In many installations, project teams may consist of 
one or two people. In such cases, the inspection 
team may consist of the moderator, the author, and 
One or two inspectors recruited from other projects. 
The author may serve at some future time on an 
inspection team that is reviewing the work done by 
the inspectors of his work, and the current modera- 
tor may, in the future, have his work inspected at a 
meeting moderated by one of the inspectors from 
other projects. 


: \ = / 


Moderator 

The moderator controls the activities of the inspec- 
tion process and acts as the manager of the inspec- 
tion meeting. To achieve impartiality, the moderator 
is usually not a member of the team that produced 


. the work product to be inspected. (If a separate 


development assurance or development quality 
control group exists, the moderator is often chosen 
from that group.) Although the moderator must 
manage the inspection process, technical personnel 
(systems analysts or programmers) rather than 
managerial personnel serve as moderators. In 
system control programming development within 
IBM, the moderator is usually a skilled programmer 
because the same person is often called upon to 
moderate all inspections for a given project. 
Throughout the inspection process, the moderator 
is responsible for efficient resource utilization and 
maximum problem detection. During the inspection 
meeting, the moderator must not allow the partici- 
pants to engage in extended disagreements, solution 
hunting, or trivial matters. Thus, it is usually neces- 
sary for the moderator to be able to use personal 
sensitivity, tact, and drive, in balanced measures, to 
conduct successful inspections. 
The moderator is responsible for the following: 
e Scheduling each step of the inspection 
e Selecting inspectors, with the assistance of the 
author and the approval of management, and 
assigning one of the inspectors to be the “‘reader”’ 
of the materials during the inspection 
e Ensuring that inspection materials have been 
distributed to the inspection team by the develop- 
ment group 
e Distributing checklists and error analysis reports 
to be used by the inspectors 
e Training all inspectors in how to prepare for and 
participate in inspection meetings 
e Planning the sequence of events for the inspection 
meeting; determining which parts of the materials 
will be inspected at given points in the meeting 
(the plan will be influenced by the kinds of 
materials, the complexity of the design, and 
whether changed design requires examination of 
other modules) 
e Recording all errors detected and all problems 
raised 
e Submitting reports after the inspection meeting 
e Evaluating the error rate and determining whether 
a reinspection is required 
_¢ Estimating the amount of rework 


Inspectors 

Typically, the inspectors include the technical 
persons responsible for the prior and succeeding 

_ development phases. As a result, inspectors for a 
detailed design inspection may include the creator of 
the initial design, the coders of the design being 
inspected, and the testers responsible for testing the 
work product. For a code inspection, the general 
and detailed designers and the testers may be the 
inspectors. The development team leader or the 
chief programmer may be an inspector (but may not 
be the moderator). When the designer and the coder 
are the same person, an inspector may be an individ- 
ual chosen from a closely related area, for example, 
the designer/coder of related components. 


Inspections and the Project Plan 

An important characteristic of the inspection process 
is that its time and costs are planned for and includ- 
ed in the overall project plan. Accordingly, when 
the project manager develops the project plan, he 
estimates the time and costs for each of the succeed- 
ing inspection steps for each inspection for each 
work product. While some projects may require 
only a few inspections, the volume of design and 
code materials involved in large projects may require 
many inspections. Estimated time and costs for 
rework are also included in the plan. 


An inspection plan is usually prepared by the 
project manager before each program development 
project. It normally lists the names of the moderator 
and the inspectors for each planned inspection, the 
work product to be inspected, an estimate of the size 
of the work product*, and an estimate of the time 
required for each step of each inspection. . 

To develop time estimates, the estimator can use a 
table based on installation experience and developed 


Inspection Step 


Initial Design 
Inspection 


Overview 500 
Preparation for inspection 100 
Inspection meeting 130 


Rework 20 hrs/K.NCSS 


Figure 7. Inspection estimating table 
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‘ by the installation’s inspection coordinator. For 


example, the table of Figure 7 shows that the 
coordinator who prepared the table expects that 150 
lines of noncommentary source statements can be 
inspected per hour during a code inspection, and that 
a code inspection meeting for a module with a size 
of 300 NCSS would take two hours. Note that only 
one overview meeting — an initial design overview 
meeting — may be necessary, especially for projects 
for which the inspectors for all phases are available 
at the time the meeting is held. For code inspec- 
tions, the overview step is usually omitted, since the 


- moderator and the inspectors are usually already 


familiar with the detailed design, as is the case when 
the code inspection team members also constitute 
the detailed design inspection team. 

The estimates in the table of Figure 7 are based on 
a system control programming environment. Appli- 
cation programming environments have experienced 
different and much higher inspection rates. There- 
fore, each installation will need to develop its own 
table based on its own experience. 

Maximums seem to be applicable for some activi- 
ties; for example, inspectors probably cannot devote 
more than four hours per day to preparation activi- 
ties, and inspection meetings seem to become much 
less efficient when their duration is greater than two 
hours. It has been found, however, that two inspec- 
tions per day (separated by other activities) can be 
successful. 


*This estimate is expressed in lines of code (LOC) or 
noncommentary source statements (NCSS), the sum of 
executable code instructions and declaratives. Instructions 
that invoke macros are counted only once, as are expanded 
macroinstructions. Comments are not counted. 


Inspection rates in NCSS/hour 


Detailed Design Code 
Inspection 


Inspection 


125 
150 
16 hrs/K.NCSS 


Planning 
Included in this initial step are the following modera- 
tor activities: 

e Ensuring that the inspection materials are distrib- 
uted to the inspection team by the development 
group 

¢ Scheduling the overview and inspection sessions 
(scheduling of the meetings should permit the 
inspectors sufficient study time and permit flexi- 
bility in their schedules) 


Overview Meeting 

The purpose of the initial design overview meeting is 
educational. The designer’s presentation should 
provide an understanding of the design’s major 
functions and functional relationships as well as a 
detailed description of the materials. The designer 
includes in his presentation the design logic, external 
linkages, module relationships, data areas, etc. The 
overview is attended by all initial design inspectors 
and also by other project personnel who need a 
reasonably detailed description of the project. 


Preparation for the Inspection Meeting 
This step of the inspection process is performed 
individually by all inspectors and the moderator. 
The objective is for them to become thoroughly 
familiar with the inspection materials so that during 
the inspection meeting they will be better able to 
find errors. The inspection materials may include 
those from an earlier development phase. For 


Error Category 


5 17 1 


Error Type 


Code Comments 
DA Data Area Usage 
DE Design Error 

EL External Linkages 


LO Logic 

MN Maintainability 
OT Other 

PE Performance 
PR Prologue 


PU Prog. Lang. Usage 
RU Register Usage 
SU Storage Usage 
Test and Branch 


Figure 8. Code inspections error analysis 


example, in preparing for a detailed design inspec- 
tion, the inspectors will need to refer to initial design 
materials. 

While examining the materials, they: 

eCheck that the materials for the work product 
being inspected match materials from the previous 
phase. For example, detailed design specifica- 
tions should not deviate from the requirements of 
the initial design specifications. 

e Understand the required inputs and expected 
outputs (external linkages to and from each 
module). 

e Understand the data area environment of each 
module. 

e Comprehend the control flow and logic. 

¢ Check that the exit criteria have been met. 

e Note discrepancies or errors found so that they 
may be recorded during the inspection meeting. 
No attempt is made during this step to find 
solutions to any problems uncovered. That is the 
function of the rework step. 

Inspectors should also familiarize themselves with 
the latest error analysis report for the type of inspec- 
tion being conducted (see Figure 8). This type of 
analysis, prepared from data gathered from previous 
inspections, helps the inspectors focus on those 
errors which occur most frequently. (In Figure 8 
these are design errors, logic errors, and prologue 
errors.) Such «“alyses also assist in the updating of 
checklists (see Figure 9) which list examples of each 
type of error. Inspectors should also use the appro- 


Sl CRAIN Si SOM 


Total 


11 


LOGIC 


| Missing 


Are all constants defined? 


Are al! unique values explicitly tested on input parameters? 


Are values stored after they are calculated? 


Are all defaults checked explicitly tested on input parameters? 

If character strings are created, are they complete? Are all delimiters shown? 

If a keyword has mar+ unique values, are they all checked? 

If a queue is being manipulated, can the execution be interrupted? If so, is queue protected by a 
locking structure? Can the queue be destroyed over an interrupt? 


Are registers being restored on exits? 
Are all keywords tested in macro? 


Are all keyword-related parameters tested in service routine? . 
Are queues being held in isolation so that subsequent interrupting requesters are receiving spurious 


returns regarding the held queue? 
Should any registers be saved on entry? 


Are all increment counts properly initialized (0 or 1)? 


Are absolutes shown where there should be symbolics? 
On comparison of two bytes, should all bits be compared? 
On built data strings, should they be character or hex? 
Are internal variables unique or confusing if concatenated? 


Are all blocks shown in design necessary or are they extraneous? 


Figure 9. A portion of a detailed design checklist 


priate checklist during the preparation step. Appen- 
dix D contains sample checklists developed within 
IBM in a system control programming environment. 
Each installation will probably wish to develop its 
own, based on its own environment and error 
experience. 


Inspection Meeting 
The inspection team and the author of the work 
product being inspected attend the inspection 
meeting. In general, others are not encouraged to 
attend, since the meeting is a working session with a 
fixed objective — to find errors. 

At the beginning of the meeting, the moderator 
describes to the group the sequence in which the 
materials are to be examined. The author’s role in 


the inspection is usually limited to answering techni-- 


cal questions. The author does not conduct the 


meeting — that is the responsibility of the moderator. 


The moderator has, before the meeting, appointed 
one of the inspectors (usually a “‘key”’ inspector) to 
“‘read’’ aloud the inspection materials. The inspec- 
tion is more effective if the reader paraphrases the 
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materials instead of reading them verbatim because 
paraphrasing tends to keep the other participants 
more alert and helps the author determine whether 
the materials can be understood. As the reading 
proceeds, each inspector looks for errors or ambigui- 
ties and for adherence to the exit criteria. The 
collective activity of group inspection tends to find 
more errors than the sum of individual inspector 
efforts. 

As errors are recognized, the moderator records 
them in a problem list, estimating the rework time 
and classifying them by error type, error category, 
and error severity: | 

« Error type — for example, logic error, disagree- 
ment of code with design specifications, lack of 
adherence to maintainability or performance | 
requirements. | 

e Error category — missing, incorrect, or extra 
design or code. 

e Error severity — major or minor. An error which 
would cause malfunction or which precludes the 
attainment of expected or previously specified 
results is considered to be a major error. 


The moderator might record the following to 
indicate a logic error (LO), categorized as incorrect 
code (W), of major severity (MAJ), with an estimated 
rework time of 20 minutes: 
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At the conclusion of the inspection meeting, the 
moderator seeks the team’s agreement with the 
correctness of the error list and decides whether a 
reinspection is required after the errors have been 
corrected. This decision is based on installation- 
established standards. For example, an installation’s 
standards may specify that detailed design materials 
may not exceed an error rate of five percent and that 
code may not contain more than one major problem 
per 25 lines of source code. 


Initial Design Inspection Meeting 

For an initial design inspection meeting, the modera- 
tor usually chooses either the team leader or the 
author of the detailed design as the inspector “‘to 
read” the design. As the “‘reading”’ proceeds, each 
inspector follows the logic looking for errors or 
ambiguities, and questioning the design logic as 
necessary. 


Detailed Design Inspection Meeting 

At a detailed design inspection meeting, the pro- 
grammer often serves as the reader. Certain users of 
inspections have the initial designer perform that 
function, believing that the initial designer is in a 
better. position to evaluate linkages to other parts of 
the system that may have been overlooked by the 
author of the detailed design. At the meeting, the 
team checks the detailed design materials for consis- 
tency with the initial design, covers every piece of 
logic at least once, and checks each design statement 


for ambiguities that could lead to coding errors. The _ 


emphasis in this inspection meeting is on detecting 
Omissions; user experience has shown that most 


problems discovered during a detailed design inspec- 


tion are design omissions. Inspectors also check for 
incorrect design and areas of overdesign (extra, 
unnecessary logic). 


Code Inspection Meeting 

At a code inspection meeting, the detailed designer 
(unless the detailed designer is also the programmer) 
usually serves as the reader. A statement-by- 
statement comparison is usually made between the 
current detailed design and the code. Every line of 
code is read and every path is checked. Code 
inspections emphasize the detection of wrong rather 
than missing or extraneous code. Correctness of 
structured code may be verified by either of the two 
following methods, although the second method is 
usually more successful: 

1. The code is traced in sequential page order, 
Starting with the main line segment, followed by 
the lower-level segments. 

2. Main line logic is traced completely through 
every subroutine until the main line logic has 
been completely traced. Then all remaining 
secondary paths are traced. 


Rework 

Within one day of the inspection meeting, the 
moderator distributes to the participants the code 
inspection module detail report (see Figure 10), 


which summarizes, and to which is attached, the 


problem list prepared by the moderator during the 
inspection meeting. The author then proceeds to 
correct the problems specified in the report and list. 
The moderator also distributes to the team and 
project management the summary inspection report 
(see Figure 11), which may contain the results of 
one inspection encompassing several modules — 
modules forming a particular part of the system. 
Note that in the summary report the moderator 
records actual hours spent for the first three inspec- 
tion steps — overview, preparation, and meeting — 
and estimated rework and follow-up hours. Manage- 
ment can compare these estimates, which are based 
on errors actually found, to the estimates in the 
project plan and determine whether the project is 
still on schedule. Note that since estimates for 
rework were included in the project plan, project 
schedules and costs are much less likely to be 
affected by any necessary rework. 

Management, then, can use the summary report to 
keep track of the project’s quality and schedule. The 
author, by comparing the detail report to the error 
analysis report (see Figure 8) prepared by the 
inspections coordinator, can determine in what areas 


to strive for improvement. 


Appendix E contains samples of the module detail 


_ reports (one for each type of inspection) and the 


summary report, along with explanations of the 
various report fields. 
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CODE INSPECTION MODULE DETAIL REPORT 


Date 


Module: 


Component/Application .. 


= 
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ERR 


Problem Type: 

LO: - Logic 

TB: Test and Branch 

EL: External Linkages 
RU: Register Usage 

SU: Storage Usage 

DA: Data Area von 

PU: Program Language Usage 
PE: Performance 

MN: Maintainability | | 
DE: Design Error 

PR: Prologue 


CC: Code Comments 


PT TT ty tt tt fe 


OT: Other 


TOTAL 


REINSPECTION REQUIRED? ___ 


Figure 10. Code inspection module detail report | 
tor simply believes that the reworked materials 


Follow-Up should be reviewed by others, a reinspection may be 
After the errors and ambiguities have been correct- scheduled at this time. : 

ed, and if a reinspection has not already been | From the error count and inspection hours 
scheduled, the moderator verifies the completeness information in the detail and summary reports, the 
and accuracy of the reworked materials and gives his coordinator (or moderator, if the installation does 
formal approval, thereby allowing the development not have a coordinator) develops the data base used 
effort to move forward. If the moderator finds that to produce the error analysis report (Figure 8) and 
the amount of rework warrants a reinspection other reports used to improve the application devel- 
(“more than five percent of the materials have been opment and inspection processes, as discussed in 


reworked” is a possible criterion), or if the modera- _ Chapter 3. .. | 7 


a. 


cl 


SUMMARY INSPECTION REPORT _ INITIAL DESIGN{_J DETAILED DESIGN (_] CODE (1 


Date 

To: Design manager Development manager 
Subject: Inspection report for. Ci‘“(‘(‘(‘“(($(;NSNNNNUUNUUUUOCC(C RNCO#SWSprecttion diate 

Application _ 

Component(s) 

Work Performed By 
Initial Detailed | Inspection Person-Hours (X.X) 
Full | Designer (CJ}Designer (7) ELOC/NCSS 


~” 
me) 


New or Detailed Programmer ' Added, Modified, Deleted 
Module| or Part | Designer (CJ Co 
Name | Mod. | Insp. Tester Cj 


Insp. | Re- Follow- 
Meetg. | work _ up 


Reinspection required?_ SC Length of. inspection (clock hours and tenths) 
Reéinspection by (date) hos Additional modules 


DCR ID’s written 


Problem summary: Major Minor Total 
Errors in changed code: Major Minor. CC:C*dErrrrcr's inn. basse COO: -- - -Major __. Minor 
Initial Designer Detailed Designer Programmer Team Leader Other Moderator’s Signature 


Figure 11. Summary inspection report 


Component 


Chapter 3. Using the Inspections Data Base 


Comparing the application development process 
again to industrial continuous flow processes, it can 
be seen that two major concepts of industrial process 
control — feedback and feedforward — can be applied 
to the application development process. In feed- 
back, a step in the process is measured with the 
measurement used to make adjustments to that step 
so that it will make a more significant contribution to 
the whole process. In feedforward, the measurement 
of a step is used to affect succeeding steps in the 
process. 

Examples of feedback and feedforward in the 

application development process are: 

¢Data about a designer’s or a programmer’s work 
product resulting from an inspection is fed back 
to him so that he can see the types of errors he 1s 
making, compared to others in the installation, 
and so produce a better quality product in his 
next assignment. 

e Data developed about a product during an inspec- 
tion can, when known to people involved in 
succeeding development phases (feedforward), 
help in improving the product. For instance, if a 
design inspection reveals that certain types of 
errors have been more common than usual, the 
tester should be prepared to design test cases that 
concentrate on that type of error. 


Module Name 


ECHO 
ZULU 
FOXTROT 
ALPHA 
LIMA 
DELTA 


Figure 12. A listing of most error-prone modules 
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Lines of Code 


A prerequisite for process control in an industrial 
process or in application development is the 
availability of data. The inspection process provides 
a disciplined technique for the collection of data 
about applications in development. When this data 
is systematically collected and analyzed, steps can be 
taken to improve those applications and the applica- 
tion development process. Some specific uses of this 
data follow. 

Identification of “‘error-prone modules’’. A listing 
of initial design, detailed design, or code design data, 
or some combination thereof, as in Figure 12, which 
combines detailed design and code design inspection 
data, highlights the modules for which the error 
density was highest. A commonly used measure of 
error density is errors per 1000 lines of noncommen- 
tary source statements, or “‘errors/K NCSS. As can 
be seen from Figure 12, this measurement is devel- 
oped using the following formula: 


Number of errors in the module 
Number of lines of noncommentary 
source statements in the module 


If the error detection efficiency of inspections is 


X 1000 


fairly constant, module and project quality can be 


predicted fairly early — certainly before unit testing — 


and steps can be taken to improve quality considered 
to be below standard. For example, if error detec- 


Error Density 
(Errors/K NCSS) 


31 
31 
28 
27 Average error density 
19 
15 


tion efficiency of a design inspection is 50 percent 
and the inspection found ten errors in a module, it 
can be estimated that there are ten errors remaining 
in the module. If this is below standard, manage- 
ment could decide to reinspect or redesign the 
module before coding it. Another option might be 
to plan an especially rigorous test for the module and 
to concentrate the testing on those errors of the type 
found in the inspection. 

Early identification of high-incidence error types. 
Comparing the distribution of error types in the first 
modules inspected for a project to the installation’s 
normal or usual distribution of errors provides an 
early warning of possible project problems and can 
make it possible to take steps to remedy the problem 
for the remaining modules of the project. Figure 13, 
for example, shows that module X contained almost 
twice as many linkage errors as is usual. Provision 
could be made to test module X and other early 
modules to remove the unusually high incidence of 
such errors. In addition, other developers may be 
warned of the situation and of possible peculiarities 
in this application, and so be better prepared to 
perform their functions for this project. 


Normal /Usual 


No. Errors Distribution % 


Logic 23 44 
Linkage 21 18 


Data Areas 6 13 
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Figure 13. Distribution of error types for module X 


Self-improvement. Developers can use a listing 
such as that in Figure 14 to concentrate their efforts 
on the areas of their own work that most need 
improvement. They can see what types of errors 
they made in the subject module and how frequently 
they occurred compared to the normal or usual 
distribution of errors in the installation. 

Improvement of the development process. Cumula- 
tive error distributions by error type for a particular 
type of inspection, as in Figure 14, can show devel- 
opment management which types of errors are most 
prevalent and thus help focus attention toward 
reducing their frequency. Such a report, may, for 


example, show overall shortcomings in the training 
given to development personnel and the need for 
changes in development standards and procedures. 
When compared to a cumulative error distribution of 
a later period, as in Figure 15, management can 
determine the progress made in reducing the fre- 
quency with which certain errors occur. 

Improvement of the inspection process. Analysis 
of inspection results may show that inspections 
conducted by a particular moderator yield signifi- 
cantly fewer major and considerably more minor 
errors per man-hour of effort when compared with 
inspections conducted by other moderators. Such 
analyses can help moderators redirect and discipline 
their efforts. 

Analysis of reports showing error distributions by 
type can assist moderators in developing checklists 
that will direct the attention of inspection teams to 
those errors that occur most frequently. 

The person hours required by inspections can be 
continually compared to installation averages to 
evaluate the inspection process. Inspections should 
not only improve the quality of programs but should 
reduce the net time required to develop them; the 
total time required by all participants for all inspec- 
tions, plus total development time, should be less 
than the development time along without 
inspections. 


Inspections Data and Personnel 
Considerations 

In extensive IBM use of inspections and some use by 
non-IBM installations, employee dissatisfaction has 
not been reported as a problem. Primarily this is 
because developers find inspections to be of assist- 
ance to them in assuring the quality of their work 
and improving their ability to meet schedules. Also, 
managers quickly learn that it is undesirable and 
unrealistic to rely solely on inspections data to 
evaluate an individual’s work quality. Experience 
has shown that management sees the necessity of 
continuing to weigh all available facts when apprais- 
ing an individual. Some of the reasons for this are: 

1. Only the quality of the final product, and not of 
intermediate stages, should be considered when 
evaluating the developer. The same principle 
applies to productivity: the overall time spent 
by the developer is much more significant than 
the time spent at any one stage. 

2. The number of errors uncovered depends 
largely upon the thoroughness of the inspection; 
the better the inspection, the greater the num- 
ber of errors detected (all other factors being 
equal). 
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| | Error Category Total 


, Code Comments 1 
Data Area Usage 

~ Design Error 14 

' External Linkages 3 

_ Logic 10 

_ Maintainability 

- Other 

' Performance 

~ Prologue 

: Prog. Lang. Usage. 

~ Register Usage 

Storage Usage 

~ Test and Branch 


3. The number of errors made by a developer is 


PERCENT affected by the complexity of the program, the 
ERROR TYPE Aug. 1 quality of prior development, the time available 
for development, the time actually spent by the 
Code Comments developer before the inspection, and differences 


Data Area Usage 

Design Error 

External Linkage 

Design Documents : ; 4 
Logic 

Maintainability 


in the modes of operation of individual develop- 
ers (such as “‘very slow and error-free”’ or “‘very 
fast and error-prone’). 

. A tendency to minimize the number of errors 
uncovered and their severity may occur if the 
errors are used to appraise the quality of work 
performed by an individual, since the members 
of the inspection team will also have their 
development work subjected to inspections. 
Thus, the value of statistics gathered during 
inspections may be largely destroyed. 

._ If errors are “punished’’, inspections may be 
delayed unduly by the developer who wants to 
detect and to correct as many errors as possible 
before the inspection to avoid having a “poor” 
record entered into the data base. . 


Other 


Performance 

Prologue/Prose 

Program Language 

Register Usage | 

Storage Usage : 5 
Test and Branch 


Figure 15. Cumulative error distribution by type 
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Chapter 4. Moderator Training Hints 


This chapter is designed to assist an installation’s 
first moderator in preparing for his role and in 
training other moderators. Subsequent moderators 
can also use this chapter when preparing to conduct 
inspections. 

The moderator-trainee should first study this text, 
including Appendixes A-E. The case study presented 
in Appendix A consists of a COBOL source listing, 
together with supporting design documents. Al- 
though it is written in COBOL, the case study has 
been used with little difficulty by people with 
programming, but no COBOL, experience. 

Then, the inspection steps for the case study 
inspection should be planned: the overview meeting, 
the preparation step, the inspection meeting, rework, 
and follow-up. 

The participants in the case study inspection 
should be selected from members of the project 
_ selected by management to be the inspections pilot 
project. As indicated in Chapter 1, under 
“Implementing Inspections’’, it is desirable for the 
moderator to report to the same manager as that of 
the pilot project, although he should not be a mem- 
ber of the pilot project team. In addition, a 
moderator-trainee may be invited to act as an 
inspector. The participants can prepare themselves 
for their first inspection by reading Chapters 1 and 2 
of this manual. 

Since the author of the case study is not avdilable, 
the moderator should appoint one of the inspectors 
to assume the role of author and present the over- 
view of the case study during the overview meeting. 

Sufficient time should be scheduled for the prepa- 
ration step, and the case study materials, including 
the COBOL checklist (Appendix D), should be 
delivered to the participants early enough so that the 
inspectors can adequately prepare for the inspection 
meeting. The guidelines suggested in Figure 7 can 
initially be used to schedule preparation time. When 
delivering the materials to the inspectors, the moder- 
ator should notify the inspector he has chosen to be 
the “‘code reader’ during the inspection meeting, 
because this person will want to make special 
preparations for that activity. 

During the inspection meeting, the moderator 
should write a short description of each error discov- 


ered, or, if this is not possible, mark the errors on 
the listing and write the descriptions after the 
meeting. 

After the program listing has been completely 
inspected, the moderator should read to the partici- 
pants his interpretation of each of the errors and 
obtain agreement that it is accurate. Then the 
moderator should make a decision as to whether a 
reinspection is required and so inform the partici- 
pants. (Since inspection standards unique to this 
installation have not yet been developed, the deci- 
sion can be based on whether there are more than 
25 major errors per 1000 lines of noncommentary 
source statements.) Finally, the participants should 
discuss the error list in the case study solution 
(Appendix B). 

After the meeting, the moderator should prepare 
the detail and summary inspection reports 
(guidelines are given in Appendix E), compare the 
reports against the samples shown in the case study 
solution, and inform management of the results of 
the case study inspection. 

Plans should then be made for the pilot project’s 
first inspection. Before holding that inspection, 
however, the moderator should, in conjunction with 
management and the installation’s standards group, — 
develop the exit criteria to be used for the pilot 
project’s inspections. The appropriate checklists in 
Appendix D may also be tailored to the installation’s 
use, as may the reporting forms shown in 
Appendix E. 

Beginning with the pilot project’s first inspection, 
the error and inspections hours data from the 
inspection detail and summary reports is entered into 
the inspections data base. Only with a conscien- 
tiously maintained data base can the benefits de- 
scribed in Chapter 3 be achieved. 

The second and subsequent moderators can learn 
the moderator role in a similar manner. In addition, 
it may be helpful for moderator-trainees to observe 
an inspection of one of the installation’s develop- 
ment projects; afterward, they can benefit from a 
critique of the session with the session moderator. 
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Appendix A: Case Study 


This case study provides materials (Figures 16, 17, 
and 18) for a code inspection of a COBOL program 
that reads student enrollment cards, checks them for 
errors, and prints out card images with errors flag- 
ged. Although the application may be somewhat 
trivial, it is of sufficient difficulty for a realistic code 
inspection. 

The materials to be used by the participants in the 
inspection follow. They consist of design docu- 
ments, a card layout form, a sample output listing, 
and a program listing. The COBOL checklist in 
Appendix D may be used during the preparation and 
inspection steps to focus the inspectors’ attention on 
common errors. Appendix B is the “‘school solution” 
of the case study. Chapter 4, “Moderator Training 
Hints’’, should be reviewed before the case study 
materials are reviewed. 


Initial Design Requirements | 
The ‘“CHECKER” program is to read cards containing 
student enrollment and grade information, validity- 
check the data on the cards, and print out the card 
images with errors appropriately flagged. More 
‘specifically: | 


SERIAL |DEPT. 
LAST NAME NO. NO. 


22 23 25 26 29 3031 323334 


Figure 16a. Enrollment and completion card layout 
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¢ Two types of data cards are processed: 
(1) completion cards, and (2) enrollment cards. 
The two types must be grouped separately in the 
input deck, with the completion cards followed by 
the enrollment cards, separated by a card with a $ 
in column 1. 

e Enrollment cards are distinguished from comple- 
tion cards by a blank in the grade field 
(column 65). 

¢ Card images are printed sequentially, with enroll- 
ment cards printed on a separate page. 

e Card fields that fail validity checking (that are in 
error) are flagged with an underscore and a 
vertical bar under the field in error. 


Detailed Design Specifications 

Figure 16a shows the layout of the enrollment and 
completion cards and Figure 16b shows the detailed 
specifications for each card field. (Any time 
“error” is indicated in these specifications, the error 
flag should be set for the column(s) in error.) Figure 
17 shows a sample report. 


COURSE TITLE COURSE! pRos}| CRSE JQ] DATE | TA DATE |¢ 
Ne: HRS |} Comp. } L/D BEGUN |p 


53 54 57 58 60 61 646566 6970717273747576 7980 


Card Column 
1-2 


Field Name and Description 
INITIALS 


If $, this is a contro] card indicating that all 
following input cards should bear no grade 
(col. 65 = blank); if not $ or a letter — error. 


If not letter or blank — error. 


LAST NAME 


. Last name must start in col. 3; if not, all leading 


blanks are flagged as errors. If not letter, blank, 
single quote, or hyphen in cols. 3-16 — error. 


SERIAL NO. 
If not W or numeric — error. 


If not numeric — error. 


DEPT. NO. 
If col. 23-25 not letters or numbers — error. 


BLDG/FLR 


1. If Record Type (col. 32-33) = 10 (Graduate 
Work Study) 


a. Cols. 26-29 must contain a valid school 
code. Valid school codes: 0293, 0946, 
1126, 1361, 6012. 


b. If cols. 26-29 not valid school code — error. 
. If Record Type not 10 


a. If LOC = K, cols. 26-28 must contain a 
valid Bldg. No. Valid Bldg. Nos.: 001,002, 
003, 004, 005, 021, 022, 023, 024, 033, 
034, 035, 042, 043, 045, 052, 057, 201, 
202, 210, 211, 212, 959, 960, 964, 965, 
966, If Bldg. No. not valid — error. 


b. If LOC = X, cols. 26-29 must contain a 
valid country code. Valid codes: AUST, 
BELG, CANA, ENGL, FRAN, GERM, 
ITAL, JAPA, NETH, SCOT, SWED, SWIT. 
If code not valid — error. 


Cc. If LOC = Z, cols. 26-29 must contain valid 
three-letter locations codes (col. 29 must be 
blank). Valid codes: BOT, LGC, LSC, PAA, 
RMD. If code not valid — error. 


d. If LOC not, K, X, or Z, cols. 26-28 must be 


a combination of blanks, numbers, or letters. 


If not blanks, numbers or letters — error. 


e. If LOC not X or Z and Floor (col. 29) not 
blank, numbers, or letters — error. 


LOC/DIV 


1. If col. 30 not valid location code — error. 
Valid LOC codes: A, B, C, D, E, F,G, H, J, 
K, L, M, N, O, P,Q, R, S, T, V, W, X, Y, Z. 

. If col. 31 not valid division code — error. 
Valid DIV codes: A, C, D, F, G, H,1,J,K,L, 
N, O, P,Q, R,S, T, W, X, Y. 


REC TYPE 


If Record Type not valid — error (col. 32 and 
col. 33). Valid Record Types: 10, 35, 40, 46, 47, 
52, 54. 


COURSE TITLE 


Title must start in col. 34. If not, all leading 
blanks are flagged as errors. 


Figure 16b. Card field detailed specifications 


Card Column 
54-57 


Field Name and Description 
COURSE NO. 


_If col. 54 not numeric (0-9) or S, F, or D — error. 


If cols. 55-57 not numeric — error. 


PROJ 


If col. 32 and col. 33 (Record Type) = 47, cols. 
58-60 must be V30 or V31. If col. 32 and col. 
33 = 46, cols. 58-60 must be V28 or V29. If 
col. 32 and col. 33 = 10, cols. 58-60 must be 
GWS. If col. 32 and col. 33 not 10, 46, or 47, 
cols. 58-60 must be V32 or V33. If any of the 
above do not match — error. 

COURSE HRS 

If cols. 61-64 not numeric — error. If cols. 
61-64 greater than 320 — error. 

GRADE 


1. If checking completion cards ($ in col. 1 
card not yet encountered), col. 65 must have 
valid grade code. Valid grade codes: A, B,C, 
D,F,1,N,0,S,T, U, W, Z. If valid grade 
code not found — error. 


. If checking enrollment cards ($ card encount- 
ered), col. 65 must be blank. If not blank — 
error. 

DATE COMP 

1. If cols. 66-69 not numeric — error. 

2. If col. 66 and col. 67 (month) greater than 
12 — error. 

TA L/D 

1. If col. 70 not K — error. 

2. If col. 71 not L — error. 


CRDS 


1. If col. 32 and col. 33 = 1, col. 72 must be 
numeric. If not — error. 


2. If cols. 32-33 not 10, col. 72 must be blank 
or numeric. If not — error. 

FLAG 

If col. 73 not blank, 1, or 2 — error. 


PM 
If col. 74 not 1-9 — error. 


B or F 
If col. 75 not blank, B, or F — error. 


DATE BEGUN 


1. If cols. 76-79 blank, bypass further checking 
of these columns. 


2. If cols. 76-79 not numeric — error. 


3. If col. 76 and col. 77 (month) greater than 
12 — error. | 


. If col. 78 and col. 79 (year begun) greater 
than col. 68 and col. 69 (year completed) 
— error. 
ACD 
If col. 80 not blank, A, C, or D — error. 
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| NAME 


| | ID DPT L RT TITLE ¢_NO HRS G TA F B : 
| 
: a ne Fo locbcwal cast El Coes 
| | | B cera | 

NLBROWN 40460363M6012KL10DIGITAL SIMULATION 6000GWSOO30A0674KL225 0474C 
| | (| 

NLBROWN 40460363M6012KL10SYS ANALYSIS 6001GWSOO030A0674KL225 0474C 
: | 1 

REGREEN 50647001L6012KL10SYS ANALYSIS 6000GWBO030A0674KL225 0474C 


Pil ltl 
KKK KKK KK KK KKK KKK KKK KER KKK KKK KK KKK KKK KEK KK KEKE KEK KKK KK KKK KKK 


THE FOLLOWING ARE ENROLLMENTS--GRADES SHOULD BE BLANK 


KREKEKKEKKEKKEKEKKEKKEKEKKEKEKKKEKEKRKKEKRKKKKKKEKEKRKK KKK KKK KEKE KKKEKKKKEKKKKKKKEK 


| NAME | ID DPT L RTI TITLE C_ NO HRS G TA F B D 

| | | BLDG!p! ! | Pry! 'come! c mM BEG ! 

I l [4 ‘ae ee | lruel 

rot rai | 1 | I ttl | 
$ end comme Perec ee 
ltl tt Zz 

VSSMITH 20781450L6012KL10STOCHASTIC MODELS OR6000GWS0030 0674KL225 0474C 
rt ttt I 

JBWHITE 41783466L6012KL10ADV COMPILER DES 6000GWSO0030 0674KL225 0474C 


Pi I! [ 
Figure 17. Checker output | 
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Page of GC20-2000-0 


Revised August 15, 1978 


By TNL GN20-3814 


PP 5740-CB1 RELEASE 2.1 OCT 11, 1977 IBM OS/VS CCBOL 
00001 IDENTIFICATION DIVISION. 

00002 PROGRAM-ID. "CHECKER'. 

00003 REMARKS. THIS PROGRAM READS CARDS CONTAINING STUDENT 

00004 ENROLLMENT AND GRADE INFORMATION, VALIDITY CHECKS 
00005 THE DATA ON THE CARDS, AND PRINTS OUT THE CARD 
00006 IMAGES WITH ERRORS FLAGGED APPROPRIATELY. 

00007 

00008 ENVIRONMENT DIVISICN. 

00009 CCNFIGURATION SECTION. 

00010 OBJECT-COMPUTER. IBM-370. 

00011 SCURCE-COMPUTER. IBM-370. 

00012 INPUIT-CUTPUT SECTICN. 

00013 FILE-CCNTROL. 

00014 SELECT ENROLLMENTS ASSIGN TO UT-S-ENROLL. 

00015 SELECT LISTING ASSIGN TO UT-S-LISTING. 

00016 

00017 DATA DIVISION. 

00018 FILE SECTION. 

00019 FD ENRCLLMENTS 

00020 LABEL RECORDS IS STANDARD 

00021 DATA RECCRD IS CARD-IN. 

00022 01 CARD-IN PIC x(80). 

00023 

00024 FD LISTING 

00025 LABEL RECORD IS STANDARD 

00026 CATA RECORD IS LINE-OUT. 

00027 01 LINE-OUT PIC xX(81). 

00028 

00029 WORKING-STORAGE SECTION. 

00030 77. EOF-IND PIC xX. 

00031 77 + GRADE-CHECK-SWITCH PIC xX. 

00032 77 CCUNTER PIC 99 USAGE COMPUTATIONAL. 
00033 77 THIS-YEAR PIC x(2). 

00034 77=«2I PIC 99 USAGE COMPUTATIONAL. 
00035 77,°=«9 PIC 99 USAGE COMPUTATIONAL. 
00036 77° «IT pic xX. 

00037 77. SPLAT PIC X(20) VALUE IS ALL "|". 
00038 01 HEADER PIC X(81) VALUE IS 

00039 "1 [ NAME | ID DPT L RI| TITLE C-NC 
00040 ' HRS G TA FB Ge. 

00041 01 HEADER2 PIC xX(81) VALUE IS 

00042 ; l | BLDUG|D| | | PR 
00043 ‘S| |COMP| C M BEG |°. 

00044 01 HEADER3 PIC X(81) VALUE IS 

00045 . | tf — ttl | 
00046 on eee ft) adl ot. 

00047 01 HEADER4 PIC xX(81) VALUE IS 

00048 MOMS ESTE EEL ELE SESE EEE EES ESS ES ESSE ETE LS EES EE EES TOS OE 
00049 01 HEADERS PIC X(81) VALUE IS 

00050 "0 THE FCLLOWING ARE ENROLLMENTS--GRADES SHCULD BE BL 
00051 "ANK'. 


Figure 18a. Checker program (1 of 13) 
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Page of GC20-2000-0 
Revised August 15, 1978 
By TNL GN20-3814 


00053 O01 CARD. | 

00054 05 INIT1 PIC xX. 

00055 05 INIT2 PIC X. 

00056 05 NAME PIC xX(14). 

00057 05 NAME-CHARACTER RECLEFINES NAME PIC xX OCCURS 14. 
00058 05 SERIAL-NC PIC X(6). 

00059 05 SERIAL-NC-CHARACTER REDEFINES SERIAL-NC PIC xX CCCURS 6. 
00060 05 DEPT~CHARACTER PIC xX OCCURS 3. 

00061 05 SCHCCL PIC xX(4). 

00062 05 OTHER-LOCATICN RECDEFINES SCHCOL PIC xX(4). 

00063 05 FILLER REDEFINES OTHER-LOCATION. 

00064 10 BLDG PIC X(3). 

00065 10 FLCOR PIC xX. 

00066 05 LOCATION PIC xX. 

00067 05 DIV PIC xX. 

00068 05 RECORD-TYPE PIC X(2). 

00069 05 TITLE PIC xX(20). 

00070 05 COURSE-NC PIC xX OCCURS 4. 

00071 05 PROJECT PIc xX(3). 

00072 05 COURSE-HCUR-VALUE PIC xX(4). 

00073 05 COURSE-HCUR-LCIGIT REDEFINES COURSE-HOUR-VALUE 

00074 PIC xX OCCURS 4. 
00075 05 GRADE PIC xX. 

00076 05 DATE-CCMPLETED. 

00077 10 MONTH-CCMPLETED PIC X(2). 

00078 10 YEAR-~CCMPLETED PIC xX(2). 

00079 05 DATE-COMPLETED-DIGIT REDEFINES CATE-CCMPLETED . 
00080 PIC xX OCCURS 4. 
00081 05 TEACHING-~LCCATION PIC xX. 

00082 05 TEACHING-DIVISION PIC xX. 

00083 05 CREDITS PIC X. 

00084 05 FLAG PIC xX. 

00085 05 PRESENTATION-MCDE PIC xX. 

00086 05 BILL-~OR-FREE PIC xX. 

00087 05 DATE-BEGUN. 

00088 10 MONTH-EEGUN PIC X(2). 

00089 10 YEAR-BEGUN PIC xX(2). 

00090 05 DATE-BEGUN-DIGIT REDEFINES DATE-BEGUN PIC xX OCCURS 4. 
00091 05 ACD PIC X. 


Figure 18a. Checker program (2 of 13) 
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00093 
00094 
00095 
00096 
00097 
00098 
00099 
00100 
00101 
00102 
00103 
00104 
00105 
00106 
00107 
00108 
00109 
00110 
00111 
00112 
00113 
00114 
00115 
00116 
00117 
00118 
00119 
00120 
00121 
00122 
00123 
00124 
00125 
00126 
00127 
00128 
00129 
00130 
00131 
00132 
00133 
00134 
00135 
00136 
00137 
00138 
00139 
00140 
00141 
00142 
00143 
00144 
00145 
00146 
00147 


01 


01 


01 


O01 


O1 


O1 


Page of GC20-2000-0 
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LOCATICN-CODES VALUE "KPFENMXZWISLHGBYVQDACJOR‘. 

05 LOCATION-CCDE PIC xX CCCURS 24 INDEXED I-LOC. 
GRADE-CODES VALUE "NSAECDFWITCUZ‘. 

05  GRADE-COCE PIC xX CCCURS 13 INDEXED I-GRADE. 
DIVISION-CODES VALUE ‘LNCWPDRACFGHIJKQSTXY'. 

05 DIVISION-CCDE PIC xX CCCURS 20 INDEXED I-DIV. 


RECORD-TYPE-CODES VALUE ‘'47465410524035'. 
05 RECORD-TYPE-COLE PIC xX(2) CCCURS 7 INDEXED I-RT. 


SCHCOL-CODES VALUE '60121361094611260293'". 
05  SCHCCL-CCDE PIC xX(4) OCCURS 5 INDEXED I-SCHCCL. 


X-~LOCATICN-CCDES VALUE ‘ENGLNETHGERMFRANCANASWECSCCTJAPASWI 
‘TAUSTBELGITAL'. 

05 X-LOCATICN-CCDE PIC xX(4) OCCURS 12 INDEXED I-XLCC. 

Z-LOCATICN-CCDES VALUE ‘RMD LSC BOT PAA IGC °. 

05 Z-LOCCATICN-CODE PIC X(4) CCCURS 5 INDEXED I-ZLOC. 


KGN-BLDGS VALUE ‘202201003211 21221000100200400502102 
"20230240340350330420430450520579599 
"60064965966". 

05 KGN-ELDG PIC X(3) OCCURS 27 INDEXELC I-BLDG. 


ERROR-LINE VALUE IS ALL SPACE. { 


05 ERR-CONTRCL PIC X. 
05 ERROR-FIELCS. 
10 FILLER | PIC X(22). 
10 ERR-CEPT PIc xX(3). 
10 ERR-SCHCOL. 
15 ERR-BLDIG PIC X(3). 
15 ERR-FLCOR PIC xX. 
10 ERR-RECORD-TYPE PIC xX(2). 
10 ERR-LCCATICN PIC xX. 
10 ERR-CIVISICN PIC kX. 
10 ERR-TITLE PIC xX(20). 
10 FILLER PIC xX(4). 
10 ERR-FRCJECT PIC xX(3). 
10 ERR-CCURSE-HOURS PIC xX(4). 
10 FILLER PIC X. 
10 ERR-MONTH-COMPLETED PIC xX(2). 
10 ERR-YEAR-COMPLETEL PIC xX(2). 
10 FILLER PIC X(6). 
10 ERR-~MONTH-PEGUN PIC xX(2). 
10 ERR-~YEAR-BEGUN PIC X(2). 
10 FILLER PIC xX. | 
05 ERR REDEFINES ERROR-FIELDS PIC xX OCCURS 80. 
DATA-LINE. 
05 CARRIAGE-CCNTROL PIC xX VALUE IS SPACE. 


05 DATA-FIELDS PIC X(80). 
05 LINE-CHARACTER REDEFINES DATA-FIELDS PIC xX OCCURS. 80. 


Figure 18a. Checker program (3 of 13) 
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00149 PROCEDURE DIVISICN. 

00150 MOVE CURRENT-DATE TO THIS-YEAR 
00151 + TWO DIGIIS RIGHT-JUSTIFIED 
00152 CPEN INPUT ENRCLLMENTS 

00153 OPEN OUTPUT LISTING 

00154 MOVE 'F' TO EFOF-IND 

00155 MOVE 'F' TO GRADE-CHECK-SWITCH 
00156 READ ENROLLMENTS INTO CARD AT END 
00157 MOVE 'N' TO EOF-IND. 

00158 PERFORM 100-MAIN-PROCESS UNTIL EOF-IND = 'N° 
00159 CLOSE ENROLLMENTS 

00160 CLOSE LISTING 

00161 STOP RUN. 

00162 

00163 100-MAIN-PROCESS. 

00164 IF INIT1 = '$* 

00165 MOVE 'N' TO GRADE-CHECK-SWITCH 
00166 WRITE LINE-OUT FROM HEADER4 
00167 WRITE LINE-OUT FROM HEADERS 
00168 WRITE LINE-OUT FROM HEADER4 
00169 MOVE 59 IC COUNTER 

00170 ELSE 

00171 IF INIT1 = SPACE OR INIT1 IS NCT ALPHAEFTIC 
00172 MOVE SPLAT TO ERR (1). 
00173 IF INIT1 NOT = ‘S! 

00174 PERFORM 200-MAIN-PROCESS-CCNTLD 
00175 PERFORM 201-MAIN-PROCESS-CCNTL 
00176 PERFCRM 202-MAIN-PROCESS-CCNTL 
00177 PERFORM 203-MAIN-PROCESS-CCNTLD 
00178 PERFORM 204-MAIN-PROCESS-CCNTL. 
00179 IF COUNTER > 58 

00180 ++ NEW-PAGE 

00181 MOVE 4 TCO COUNTER 

00182 WRITE LINE-OUT FROM HEADER 
00183 WRITE LINE-CUT FROM HEADER2 
00184 WRITE LINE-OUT FROM HEADER3 
00185 WRITE LINE-OUT FROM HEADER3. 
00186 #* PRINT 

00187 MOVE CARD TC DATA-FIELDS 

00188 WRITE LINE-CUT FROM DATA-LINE 
00189 ADD 1 TO COUNTER 

00190 IF ERROR-LINE NOT = SPACE 

00191 PERFORM 205-EUILD-UNDERSCORE-LINE VARYING I FROM 1 BY 1 
00192 UNTIL I > 80 

00193 + + OVERPRINT 

00194 MOVE "+" TO CARRIAGE-CONTROL 
00195 WRITE LINE-OUT FROM DATA-LINE 
00196 MOVE SPACE TC CARRIAGE-CONTROL 
00197 WRITE LINE-OUT FROM ERRCR-LINE 
00198 ADD 1 TC CCUNTER. 

00199 READ ENROLLMENTS INTO CARD AT END 
00200 MOVE "N' TO FOF-INC. 

00201 205-BUILD-UNDERSCORE-LINE. 

00202 IF ERR (I) = SPACE 

00203 MOVE SPACE TO LINE-CHARACTER (I) 
00204 ELSE 

00205 MOVE *|' TO LINE-CHARACTER (1). 


Figure 18a. Checker program (4 of 13) 


00207 
00208 
00209 
00210 
00211 
00212 
00 213 
00214 
00215 
00216 
00217 
00218 
00219 
00220 
00221 
00222 
00223 
00224 
00225 
00226 
00227 
00228 
00229 
00230 
00231 
00232 
00233 
00234 
00235 
00236 
00237 
00238 
00239 
00240 
00241 
00242 
00243 
00244 
00245 
00246 
00247 
00248 
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200-MAIN-PROCESS-CCNTD. 


* * 


** 


IF INIT2 IS NCT ALPHAPRETIC 
MOVE SPLAT TC ERR (2). 
EXAMINE NAME TALLYING LEADING SPACES 
IF TALLY NCT = ZERO 
ADD 2 I GIVING J 
PERFORM 300-POINTER-INSERT VARYING I FRCM 3 BY 1 
UNTIL I > J. 
FERFORM 301-NAME-CHECK VARYING I FROM 1 BY 1 UNTIL I > 13 
FIRST-~CHARACTER~CHECK 
MCVE SERIAL-NO-CHARACTER (1) TC IT 
IF IT IS NUMERIC OR IT = ‘'W' 
NEXT SENTENCE 
ELSE 
MOVE SPLAT TC ERR (17). 


PFERFCRM 302-SERIAL-NO-CHECK VARYING I FROM 2 BY 1 UNTIL I > 6 


CEPARTMENT-CHECK 
PERFORM 303-CEPT-CHECK VARYING I FROM 1 BY 1 UNTIL I > 3. 


300-POINTER-INSERT. 


MCVE SPLAT TC ERR (1). 


301-NAME-CHECK. 


MOVE NAME-CHARACTER (I) TC IT 

IF IT IS ALPHAPETIC CR IT = SPACE OCR QUOTE OR *-‘ CR °. 
NEXT SENTENCE 

ELSE 
ADD 2 I GIVING J 
MOVE SPLAT TC ERR (J). 


302-SERIAL—~NC-CHECK. 


IF SERIAL-NC-CHARACTER (I) IS NOT NUMERIC 
COMPUTE J = I + 17 
MOVE SPLAT TC ERR (J). 


303-CDEPT-CHECK. 


MOVE DEPTI-CHARACTER (I) TC IT 


IF IT IS NUMERIC OR IT IS ALPHARPETIC AND IT IS NOT = SPACE 


NEXT SENTENCE 

ELSE 
ADE 22 I GIVING J 
MOVE SPLAT TC ERR (J). 


Figure 18a. Checker program (5 of 13) 
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00250 201-MAIN-PROCESS-CCNTD. 

00251 IF RECORD-TYPE = '10' 

00252 PERFORM 304-SCHOOL-CHEFCK 

00253 ELSE 

00254 PERFCRM 305-RECORD-TYPE-CHECK 
00255 PERFORM 306-LOC-AND-BLDG-CHECK 
00256 IF FLOCR IS NUMERIC OR FLCCR IS ALPHABETIC 
00257 NEXT SENTENCE 

00258 ELSE 

00259 MOVE SPLAT TO ERR-FICOR. 
00260 

00261 304~SCHOCL-CHECK. 

00262 SET I-SCHCOL TO 1 

00263 SEARCH SCHCCL-COLE 

00264 AT END 

00265 MOVE SPLAT TO ERR-SCHCOL 
00266 WHEN SCHCCL = SCHOCOL-CCDE (I-SCHCOL) 
00267 NEXT SENTENCE. 

00268 

00269 305-RECORD-TYPE-CHECK. 

00270 SET I-RT TO 1 

00271 SEARCH RECCRD-TYPE-CODE 

00272 AT END 

00273 MOVE SPLAT TO ERR-RECCRD-TYPE 
00274 WHEN RECCRD-TYPE = RECCRD-TYPE-CCDE (I-RT) 
00275 NEXT SENTENCE. 

00276 

00277 306-LOC-AND-BLDG-CHECK. 

00278 IF LOCATION = ‘K'° 

00279 PERFORM 400-K-~ELDG-CHECK 

00280 ELSE IF LOCATICN = ‘X' 

00281 PERFORM 401-X-LOC-CHECK 

00282 ELSE IF LOCATICN = ‘'Z‘ 

00283 PERFORM 402-Z-LOC-CHECK 

00284 ELSE 

00285 : PERFORM 403-AN-BLDG-CHECK. 


Figure 18a. Checker program (6 of 13) 
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00287 
00288 
00289 
00290 
00291 
00292 
00293 
00294 
00295 
00296 
00297 
00298 
00299 
00300 
00301 
00302 
00303 
00304 
00305 
00306 
00307 
00308 
00309 
00310 
00311 
00312 
00313 
00314 
00315 


Figure [8a. 
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4OQO-K-PLDG-CHECK. 


SET I-BLUOG TC 1 
SEARCH KGN-BLDG 
AT END 
MOVE SPLAT TO ERR-BLDG 
WHEN BLDG = KGN-BLLEG (I-BLDG) 
NEXT SENTENCE. 


401-X-LOC-CHECK. 


SET I-XLCC TC 1 
SEARCH X-LOCATION-CODE 
AT END 
MOVE SPLAT TO ERR-~SCHCCL 
WHEN CTHER-LOCATION = X-LCCATICN-CODE (I-XLOC) 
NEXT SENTENCE. 


4Q2-Z-LOC-CHECK. 


SET I-ZLCC TC 1 
SEARCH Z-LOCATION-CODE 
AT END 
MOVE SPLAT TO ERR-SCHCOL 
WHEN OTHER-LCCATICN = Z-LCCATION-CODE (I-ZLOC) 
NEXT SENTENCE. 


40 3-AN-BLDG-CHECK. 


IF BLDG IS ALPHAFETIC OR BLUG IS NUMERIC 
NEXT SENTENCE 

ELSE 
MOVE SPLAT TC ERR-BLDG. 


Checker program (7 of 13) 
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00317 
00318 
00319 
00320 
00321 
00322 
00323 
00324 
00325 
00326 
00327 
00328 
00329 
00330 
00331 
00332 
00333 
00334 
00335 
00336 
00337 
00338 
00339 
00340 
00341 
00342 
00343 
00344 
00345 
00346 
00347 
00348 
00349 
00350 
00351 
00352 
00353 
00354 
00355 
00356 
00357 
00358 
00359 
00360 
00361 
00362 
00363 
00364 
00365 
00366 
00367 
00368 
00369 


Figure 18a. 
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202-MAIN-PROCESS-CCNID. 
SET I-LOC TO 1 
SEARCH LOCATICN-CODE 
AT END | 
MOVE SPLAT TO ERR-LOCATION 
WHEN LOCATION = LOCATICN-CCDE (I-LOC) 
NEXT SENTENCE. 
SEARCH DIVISICN-COLE | 
AT END 
MOVE SPLAT TO ERR-DIVISION 
WHEN DIV = DIVISION-CCDE (I-DIV) 
NEXT SENTENCE. | 
EXAMINE TITLE TALLYING LEADING SPACES 
IF TALLY IS NOT ZERO 
COMPUTE J = TALLY + 34 
PERFORM 300-FOINTER-INSERT VARYING I FRCM 1 BY 
UNTIL I > J. 
IF COURSE-NO (1) IS NUMERIC 
NEXT SENTENCE 
ELSE IF COURSE-NC (1) = 'S*‘ CR ‘F* OR '‘D' 
NEXT SENTENCE 
ELSE | 
MOVE SPLAT TC ERR (54). 
PERFORM 307-CCURSE-NO-CHECK VARYING I FRCM 2 EY 1 
UNTIL I > 4. 


307-CCURSE-~NC-CHECK. 
IF COURSE-NC (I) IS NCT NUMERIC 
COMPUTE J = I + 53 
MOVE SPLAT TC ERR (J). 
IF RECORD-TYPE = °47* 
IF PROJECT = ‘'V305 OR ‘'V31'" 
NEXT SENTENCE | 
FLSE 
MOVE SPLAT TO ERR-~PROJECT 
ELSE 
IF RECORCD-TYPE = ‘10’ 
IF PRCJECT = ‘GWS‘ 
NEXT SENTENCE 
ELSE 
MOVE SPLAT TO ERR~PROJECT 
ELSE 
IF PRCJECT = ‘V32" OR ‘'V33' 
NEXT SENTENCE 
ELSE 
MOVE SPLAT TO ERR-PROJECT. 
PERFORM 404-CCURSE-HOUR-CHECK VARYING I FRCM 1 BY 1 
UNTIL I > 4. 


4Q04~-COURSE-HCUR-CHECK. 
IF COURSE-HOUR-DIGIT (I) IS NCT NUMERIC 
COMPUTE J = I + 60 
MOVE SPLAT TC ERR (J). 


Checker program (8 of 13) 
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00371 203-MAIN-PROCESS-~-CCNTD. 

00372 ** COURSE-HCUR-VALUE-CHECK 

00373 IF COURSE-HOUR-VALUE > 230 

00374 MOVE SPLAT TO ERR-COURSE-HCURS. 
00375 ¥% GRADE-CHECK~SWITCH 

00376 IF GRADE-CHECK-SWITCH = ‘'F* 

00377 PERFCRM 308-COMPLETION-~GRADE-CHECK 
00378 FLSE 

00379 ++ ENROLLMENT-GRACE-CHECK 

00380 IF GRADE NCT = SPACE 

00381 MOVE SPLAT TO ERR (65). 

00382 +* CATE CHECK 

00383 PERFORM 309-CATE-CIGIT-CHECK VARYING I FRCM 1 BY 1 
00384 UNTIL I > 4 

00385 ** MONTH- CHECK 

00386 IF MCNTH-CCMPLETED > ‘12' 

00387 MOVE SPLAT TC ERR-~MONTH-CCMPLETECL. 
00388 IF TEACHING-LCCATION NOT = ‘K' 

00389 MOVE SPLAT TC ERR (70). 

00390 IF TEACHING-LCCATICN NCT = 'L' 

00391 MOVE SPLAT TC ERR (71). 

00392 IF RECORD-TYPE = ‘10° 

00393 IF CREDITS IS NOT NUMERIC 

00394 MCVE SPLAT TO ERR (72) 

00395 ELSE 

00396 NEXT SENTENCE 

00397 ELSE 

00398 IF CREDITS IS NOT NUMERIC AND CREDITS NOT 
00399 MOVE SFLAT TO 

00400 IF FLAG IS = * * CR 

00401 NEXT SENTENCE 

00402 FLSE 

00403 MOVE SPLAT TC ERR (73). 

00404 IF PRESENTATICN-MOCE IS NCT NUMERIC 
00405 CR PRESENTATION-MCDE = ZERO 
00406 MOVE SPLAT TC ERR (74). 

00407 IF BILL-CR-FREE IS = ' ' CR ‘'B' OR 'F® 
00408 NEXT SENTENCE 

00409 ELSE 

00410 MOVE SPLAT TC ERR (75). 

00411 

00412 308-CCMPLETICN-GRADE-CHECK. 

00413 SET I-GRADE TO 1 

00414 SEARCH GRADE-CCCDE 

00415 AT END 

00416 MOVE SPLAT TO ERR (65) 

00417 WHEN GRADE = GRADE-CODE (I-GRALCE) 
00418 NEXT SENTENCE. 

00419 

00420 309-CATE-DIGIT-CHECK. 

00421 IF DATE-COMPLETEDC-CIGIT (I) IS NOT NUMERIC 
00422 COMPUTE J = I + 65 

00423 MOVE SPLAT TC ERR (J). 


Figure 18a. Checker program (9 of 13) 
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00425 204-MAIN-PROCESS-CCNTD. 

00426 IF DATE-BEGUN = SPACE 

00427 NEXT SENTENCE 

00428 ELSE 

00429 PERFORM 310-CATE-BEGUN-CHECK. 
00430 IF ACD = ' *" CR ‘'C* OR ‘'D* OR ‘A! 
00431 NEXT SENTENCE 

00432 ELSE 

00433 MOVE SPLAT TC ERR (80). 

00434 

00435 310-CATE-BEGUN-CHECK. 

00436 PERFORM 405-DATE-BEGUN-DIGIT-CHECK VARYING I FROM 1 BY 1 
00437 UNTIL I > 4 

00438 IF MCNTH-BEGUN > '12' 

00439 MOVE SPLAT TC ERR-MONTH-BEGUN. 
00440 IF YFAR-BEGUN > YEAR-COMPLETED 

00441 MOVE SPLAT TC ERR-YEAR-BEGUN. 
00442 

00443 405-CATE-BEGUN-DIGIT-CHECK. 

OO444 IF DATE-BEFGUN-DIGIT (I) IS NOI NUMERIC 
00445 ; COMPUTE J = I + 75 

00446 7 MOVE SPLAT TC ERR (J). 


Figure 18a. Checker program (10 of 13) 
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DATA NAMES 


ACE 

BILL-OR-FREE 

BLDG 

CARL 

CARI-IN 
CARRIAGE-CCNTRCL 
COUNTER 
COURSE-HOUR-DIGIT 
COURSE-HOUR-VALUE 
COURSE-NO 
CRELITS 
DATA-FIELDS 
DATA-LINE 
DATE-BEGUN 
DATE-BEGUN-DIGIT 
DATE-COMPLETED 
DATE-COCMPLETED-DIGIT 
DEPT-~CHARACTER 
DIV 

DIVISION-CCDE 
DIVISION-CCDES 
ENRCLLMENTS 
EOF-IND 

ERR 


ERR-ELDG 
ERR-CONTRCL 
ERR-COURSE-HCURS 
ERR-CEPT 
ERR-CIVISICN 
ERR-FLCOR 
ERR~LOCATICN 
ERR-MONTH-BEGUN 
ERR~MONTH-CCMPLETED 
ERR-PROJECT 
ERR~RECORE-TYPE 
ERR-SCHOCL 
ERR-TITLE 
ERR~YEAR-BEGUN 
ERR~YEAR-COMPLETED 
ERRCR-FIELDS 
ERRCR-LINE 


GRATE 
GRACE-CHECK~SWITCH 
GRADE-CCDE 
GRALE-CODES 

HEADER 

HEALER2 

HEALER3 

HEALER4 

HEALERS 

I 


I-BLDG 
I-CIVv 
I-GRADE 
I-LCC 
I-RT 
I-SCHOOL 
I-xXLCC 
I-ZLCC 
INIT1 


CEFN 


000091 
000086 
000064 
000053 
000022 
000145 
000032 
000073 
000072 
000070 
000083 
000146 
000144 
000087 
000090 
000076 
000079 
000060 
000067 
000100 
000099 
000014 
000030 
000142 


000126 
000121 
000134 
000124 
000130 
000127 
000129 
000139 
000136 
000133 
000128 
000125 
000131 
000140 
000137 
000122 
000120 
000084 
000065 
000075 
000031 
000097 
000096 
000038 
000041 
000044 
000047 
000049 
000034 


000118 
000100 
000097 
000094 
000103 
000106 
000110 
000113 
000054 


Figure 18a. Checker program (11 of 13) 


CRCSS-REFERENCE DICTICNARY 


REFERENCE 
000430 

000407 

000291 000312 
000156 000187 
000156 000199 
000194 000196 
000169 000179 
000367 

000373 

000334 000336 
000393 000398 
000187 

000188 000195 
000426 

oood4u uy 

000421 

000243 

000326 

000326 

000152 000156 
000154 000157 
000172 000202 
000369 000381 
000423 000433 
000291 000315 
000374 

000326 

000259 

000321 

000439 

000387 

000351 000357 
000273 

000265 000299 
000441 

000190 000197 
000400 

000256 

000380 000416 
000155 000165 
000416 

000182 

000183 

000184 000185 
0001€6 000168 
000167 

000191 000202 
000234 000238 
000367 000368 
000288 000291 
000326 

000413 000416 
000318 000321 
000270 000273 
000262 000265 
000296 000299 
000304 000307 
000164 000171 


000199 


000181 


000344 


000159 
000158 
000209 
000389 
000446 


000362 


000307 


000376 


000203 
000239 
000383 


000173 


000189 


000199 
000200 
000221 
000391 


000205 
000243 
000421 


000394 000399 000403 
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000248 
000406 


000222 
000344 
000445 


000339 


000410 


000224 
000345 
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000346 
000416 


G00230 
000363 
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INIT2 
It 
J 


KGN-ELDG 
KGN~-ELDGS 
LINE-CHARACTER 
LINE-OUT 
LISTING 


LOCATION 
LOCATION-CODE 
LOCATION-CODES 
MONTH-BEGUN 
MONTH-COMPLETED 
NAME 
NAME-CHARACTER 
OTHER-LOCATION 
PRESENTATICN-MCDE 
PRCJECT 
RECORD-TYPE 
RECCRD-TYPE-CODE 
RECCRD-TYPE~-CODES 
SCHCOL 
SCHCOL-CODE 
SCHCCL-CODES 
SERIAL-NO 


SERIAL-~NC-CHARACTER 


SPLAT 


TEACHING-DIVISION 
TEACHING-LCCATION 
THIS-YEAR 

TITLE 
X-LCCATICN-CCDE 
X~LCCATICN-CODES 
YEAR-~BEGUN 
YEAR~COMPLETED 
Z~LCCATICN-CCDE 
Z~LCCATION-COCDES 


000055 
000036 
000035 


000118 
000115 
000147 
000027 
000015 


000066 
000094 
000093 
000088 
000077 
000056 
000057 
000062 
000085 
000071 
000068 
000103 
000102 
000061 
000106 
000105 
000058 
000059 
000037 


000082 
000081 
000033 
000069 
000110 
000108 
000089 
000078 
000113 
000112 


Figure 18a. Checker program (12 of 13) 
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000208 
000217 
000212 
000345 
000291 


000203 
000166 
000153 
000195 
000278 
000321 


000438 
000386 
000210 
000230 
000299 
000404 
000348 
000251 
000273 


000265 
000265 


000217 
000172 
000291 
000362 
000406 


0003€&8 
000150 
000329 
000299 


000440 
000440 
000307 


000218 
000213 
000346 


000205 
000167 
000160 
000197 
000280 


000307 


000354 
000273 


000238 
000209 
000299 
000369 
000410 


000390 


000230 
000234 
000368 


000168 
000166 


000282 


000359 
000347 


000221 
000307 
000374 
000416 


000231 
000235 
000369 


000182 
000167 


000321 


000353 


000227 


000315. 


000381 
000423 


000243 
000239 
000422 


000183 
000168 


000392 


000235 
000321 
000387 


000433 


000244 
000240 
000423 


000184 
000182 


000240 
000326 
000389 
000439 


000247 
000445 


000185 
000183 


000248 
000339 
000391 
000441 


000248 
000446 


000188 
000184 


000259 
000346 
000394 
000446 


000331 


000195 
000185 


000265 
000351 
000399 


000332 


000197 
000188 


000273 
000357 
000403 


PRCCEDURE NAMES 


100-MAIN-PROCESS 
200-MAIN-PROCESS-CCNILC 
201-MAIN-PROCESS-CCNTD 
202-MAIN-PRCCESS-CCNID 
203-MAIN-PROCESS-CCNTD 
204-MAIN~PROCESS-CCNTID 
205-BUILCD-UNDERSCOCRE-LINE 
300-POINTER-INSERT 
301-NAME-CHECK 
302-SERIAL~NC-CHECK 
303-DEPT-CHFCK 
304-SCHCCL-CHECK 
305-RECCRD-TYPE-CHECK 
306-LOC-AND-BLDG-CHECK 
307-COURSE-NO-CHECK 
308-COMPLETICN-GRADE-CHECK 
309-CATE-CIGIT-CHECK 
310-CATE-BEGUN-CHECK 
400-K-ELDG-CHECK 

4 01-X-LCC-CHECK 
4&02-Z-LOC-CHECK 
403-AN-BLDG-CHECK 
4&04-COURSE-HCUR-CHECK 
405-CATE-BEGUN-DIGIT-CHECK 


Figure 18a. Checker program (13 of 13) 


CEFN 


000163 
000207 
000250 
000317 
000371 
000425 
000201 
000226 
000229 
000237 
000242 
000261 
000269 
000277 
000343 
000412 
000420 
000435 
000287 
000295 
000303 
000311 
000366 
000443 


000158 
000174 
000175 
000176 
000177 
000178 
000191 
000213 
000215 
000222 
000224 
000252 
000254 
000255 
000340 
000377 
000383 
000429 
000279 
000281 
000283 
000285 
000363 
000436 


REFERENCE 


000332 


Page of GC20-2000-0 
Added August 15, 1978 
By TNL GN20-3814 


Page of GC20-2000-0 
Added August 15, 1978 
By TNL GN20-3814 


CHECKER 


100- 


MAIN-PROCESS 


203- 204- 205-BUILD- 


202- 
MAIN-PROCESS- 
CONTD 


201- 
MAIN-PROCESS- 
CONTD 


200- 
MAIN-PROCESS- 
CONTD 


MAIN-PROCESS- MAIN-PROCESS- UNDERSCORE- 
CONTD CONTD LINE 


300- 301- 303- 
POINTER- NAME- DEPT- 
INSERT CHECK CHECK 


401- 402- 
X-LOC- Z-LOC- 
CHECK CHECK 


Figure 18b. Checker program structure chart corresponding to COBOL procedure names cross-reference dictionary 


Re Pes. 
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Appendix B: Case Study Solution 


This appendix contains the error list (Figure 19) and 
detail and summary inspection reports (Figure 20 
and 21). Note that the error list (the average inspec- 
tion team finds 13 or 14 of these errors) could be 
expanded to include violations of installation 
standards. For example, the following might be an 
additional error that could be listed: 


N/W/MIN — Yine3: SPLAT AU mot 
MN /W/ . ) SH, 


(. PRIMIMIN — 


2. DA/W/MAJ — 
3. PU/W/MAJ — 


4. LO/M/MAJ —- 
5. LO/M/MAT —- 
6. DAIW/MAT - 
7.LO/W/MAJ — 
G. LO/W/MAT — 
9, PU/E|MIN — 
10. DE/w/MAT — 
I]. CO/W/MAT— 
12: LO/W/MIN— 


13: LO/W/MAT - 


14, LO/u//MAT- 


15. Lo/m [MAT — 
16. CO/w//mAT- 


17. PU/W/MAT - 


Ancor, meedy 

Line 129: ERR-RECORD-TYPE cw out of Aeguenee. 

dime 151: Be rma Lyte op an uight-Ayte field Ceurrente 
Ayte ) 


Line 3: The statement of the prologue vw the REMARKS 
PE PRNELON: 


dete) pre pnoved wnto- the two- pil (thew year). 


Lime 179: COUNTER wv mot imitcabineh at the ptart of the 
progam, Hua, the proceosng at Lune |7F ev mot 
guarantee. Le be. correct. 

dine 198: ERROR-LINE ev mevev howaebept. 

tine 205: The umdroeore. symbol shorth Le ward inatesh 
of Ae potieal ban, 

tue 212: White counting. Ake pmumbey Healing. 

jini NAME, zhe rvrong variable RS a eH ET ee 
time 2/5: NAME- CHECK cv PERFORMED one Law Lime thaw 


time 231: dw NAME- CHECK, the check for SPACE <w 
redumnibent. 


tine £3]: She design dhould. abou AR O-LMIILVER 

Line 239: bn SERIAL-NO-CHECK, the error flag iw tet into 

tine 256: Dia Lome io syeoutid_ Aoratuonas"X" and. 

'Z" NH shout Oe oe 

tine 312) The robe chota mot mateh the fe On, 
prbination alpha, Abanko , ank membherw 


tine 332: i the Arading- pacts, ont Lor many. 
hue 353: Record Agge +6 hogan pu midsing.. 
Limes3 47 through 364 pheutldh. fellou Line 341. 


time 373: the penversion. of the sonatant ty- COBOL 


DotohL puwwrk time iv eotimatide tr he 8.0 howrw. 
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Module: CHECKER 


LO: 


TB: 


EL: 


RU: 


SU: 


DA: 


PU: 


PE: 


MN: 


DE: 


PR: 


CC: 


OT: 


Logic 

Test and Branch 
External Linkages 
Register Usage 
Storage Usage 
Data Area Usage 
Program Language Usage 
Performance 
Maintainability 
Design Error 
Prologue 

Code Comments 


Other 
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CODE INSPECTION MODULE DETAIL REPORT 


REINSPECTION REQUIRED? Y 


Figure 20. Checker program code inspection module detail report 


Date 


Component/Application 


S 
> 
KH. 
O 
ws) 
= 
za 
O 
25) 


TOTAL 


CECE 
s COCO s RE 
COP eee 
COREE ee 
a 
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SUMMARY INSPECTION REPORT INITIAL DESIGN(C_J DETAILED DESIGN [7] CODE & 


To: Design manager g PA diy (0 Cc Development manager 
Subject: Inspection report for C ECK1 Inspection date Ll f / 9/ ei 


Application 


Component(s) 


Work Performed By 
Initial Detailed Inspection Person-Hours (X.X) 


Full | Designer (J|Designer (7) | ELOC/NCSS | Estimated | 


New or Detailed aly ere Added, Modified, Deleted . 
Module] or Part | Designer Co . Pre. Insp. Follow- 
. | Meetg. en 


Name | Mod. Component 


Reinspection required? YES Length of inspection (clock hours and tenths) 


Reinspection by (date) If / 25/ =. Additional modules NO 


| DCR ID's written C-2 


| Problem summary: Major ee Minor 4 ___ Total heh 


Errors in changed code: Major i Errors in base code: Major Minor 
iaesony Des Opens 
Initial Designer Detailed Designer Programmer Team Leader Other Moderator’s Signature 


Figure 21. Checker program summary inspection report 
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Appendix C: Exit Criteria 


Exit criteria are requirements to be met before a 
phase is considered complete. One objective of 
inspections is to determine whether these 
installation-established requirements have, in fact, 
been met. The initial design, detailed design, and 
code inspection exit criteria that follow are samples 
developed within IBM for a system control program- 
ming environment. Each installation should develop 
its own criteria to meet the needs of its specific 
application development environment. 


Initial Design Exit Criteria 
Initial design specifications are divided into two 
Sections: 

1. Specifications that describe the new functions as 
it applies to all affected areas of a component 
and system (Group I| specifications) 

2. Individual specifications for each module 
affected by the new function (Group II 
specifications) 

The exit criteria for Group I and Group II specifi- 

cations follow. 


Group I Specifications 

1. Completed external specifications, that is, those 
that define a function from a user or outside 
viewpoint. 

2. Internal specifications containing the following 
for all new or changed functions for each affect- 
ed component. 

a. Component Description 
An overall description of the new function — 
what is to be done. 
b. Design Rationale 
Statement of design requirements — why it is 
-to be done. Include performance and storage 
requirements. 
c. Control Flow 
Diagram showing transfer of control between 
divisions of the component. 
d. Dependencies 
e Internal 
Dependencies on other components within 
the system. 
e External 
Dependencies outside the system, such as 
hardware. 
e. Data Areas 
Names of data areas that are required by this 
function. 
f. Internal Macros 
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Names of macros contained within the system 

that are required by this function. 
g. Linkages 

e Intercomponent 
List of linkages between components re- 
quired by this function. 

e [ntracomponent 
List of linkages between major divisions of a 
component required by this function. 

3. Internal specifications containing the following 
for all new or changed functions for each affect- 
ed major division of a component. 

a. Major Division Description 
An overall description of the function as it 
affects each major division. 

b. Module List 
A list of all new or changed modules that are 
affected by the new function. 

c. Control Flow 
A graphic representation of the control flow 
(linkages) between the modules within the 
division that are affected by the new 
function. 

d. Packaging Requirements 
Any paging or link-editing required by the 
new function must be defined and listed. 


Group II Specifications 

The Group II portion of the initial design specifica- 
tions is a module description. Its purpose is to 
provide a functional description design statement 
that will be used as the source for detailed design. 
The Group II specifications are not intended for use 
as coding specifications. The following are required 
for each module affected by a new function: 

1. Functional description statements must be to a 
level of detail in the approximate range of 
15-25 source language lines of code per design 
statement. (A design statement can be ex- 
pressed in any design language, for example, 
HIPO, flowcharts, pseudo code, etc.) 

2. If a specific sequence of processing or control 
flow in a module is required, it must be stated. 

3. All required data area changes/additions must 
be specified to the field description level (field 
names and storage attributes will be defined 
during detailed design). 

4, All required linkage changes/additions must be 
defined: 

a. For macros internal to the component, 
definition must be to the macro description 
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level. (Specific macro keywords and parame- 
ters will be defined during detailed design.) 

b. For control flow between modules, all linkage 
information requirements must be defined. 
(Specific return codes, parameter formats, 
etc., are not required; they will be defined 
during detailed design.) 


. All new/changed module attributes must be 


specified: 

a. Reenterable, reusable 

b. Protection key 

c. Fixed or pageable 

d. Supervisor or problem state 
e. Enabled or disabled state 


. All new/changed performance and storage 


criteria must be specified. These include stor- 
age size and path length requirements. 


Detailed Design Exit Criteria 


1. 
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Completed specifications represent design to 
the approximate range of three to ten source 
language lines of code per design statement. (A 
design statement can be expressed in any design 
language, for example, HIPO, flowcharts, pseudo 
code, etc.) 


. Design specifications must be structured (all 


applicable structured programming rules are to 
be followed). 


. For design specifications covering modified 


modules, all new/changed statements should be 
flagged with release identification. 


. References to data areas should be by field 


name. 


. Macro invocation should specify all required 


parameters and associated values. Those pa- 


rameters that are not specified are assumed to | 
be default values. | 


. Linkages to external routines/modules must be 


defined. Parameter values passed and return 
codes expected are specified. 


. Messages issued by modules must be defined. 


Identification numbers and message texts are 
specified. Entry and exit points from modules 
must be defined. 


. All new/changed data areas must be document- 


ed in respective design documentation. 


. Any design changes to initial design since initial 


design exit must be included in the current 
design specifications; that is, initial and detailed 
design materials must match. 


Code Inspection Exit Criteria 


1. 


Complete and up-to-date module prologue. 


2. Error-free compile (that is, no higher-level 


3. 


messages than warning messages). 

Program listing containing: 

a. Compile phase output with cross-reference 
listing 

b. Assembler language (BAL) phase output with 
cross-reference listing 

c. Data area maps 

d. New/changed lines of code flagged with 
programmer identification code 


. Any design changes since detailed design exit 


must be included in current design specifica- 
tions; that is, design and code materials must 
match. 


Appendix D: Checklists 


Checklists provide the inspection team with a set of 
prompters to help uncover high-cost, high- 
occurrence errors in the work product being inspect- 
ed, and are based on an analysis of errors found 
during prior inspections. The inspectors review the 
appropriate checklists during the preparation period 
and also use them during the inspection meeting to 
help them focus on the error types listed. 

The initial design, detailed design, and code 
inspection checklists that follow in this appendix are 
samples developed in a system control programming 
environment. Each installation will develop its own 
checklists to reflect its own error statistics and 
requirements. Note that the errors specified in the 
sample checklists are associated with a two-character 
alphabetic error type code that corresponds to the 
codes used on the inspection report forms (see 
Appendix E). 


Initial Design Inspection Checklists 

The Group I and Group II checklists are used with 
Group I and Group II initial design specifications, 
respectively (see Appendix C). 


Group I Checklist 

1. Is the design consistent with the program 
objectives or requirements? ES 

2. Is the external specification for all new/changed 
externals correct and in sufficient detail? ES 

3. Does the external specification properly address 
human factors considerations (for example, 
eliminate redundant user options, create mean- 
ingful and usable user macros, keywords, 
etc.)? ST 

4. Do the Group I specifications give an accurate 
and complete: SC 

a. Overall description of the new function? 

b. Graphic description of the new function, 
showing control flow between linking mod- 
ules, both within and outside the function 
(component and system)? 

c. List of external dependencies? 

d. List of packaging requirements (modules that 
must be link-edited or paged together)? 

e. List of control blocks and a description of 
new functions for which they are used? 

f. List of component macros required by new 
function? 


Group II Checklist 
1. Do the Group II specifications give a complete 
and accurate description of the overall function 


of the module? 

a. Is all new/changed design specified at the 
functional description level? SC 

b. Is the design understandable as stated? Can 
detailed design be implemented from this 
level of detail? SC 

c. Does the functional description cover all 
known possible cases (for example, no miss- 
ing logic)? Watch in particular for exception 
cases. LO 

d. Does the functional description cover abnor- 
mal conditions (for example, what would 
happen if there is an ABEND)? ST 

e. For design being implemented in more than 
one system, does the new function fit correct- 
ly into all systems (each system may present 
unique situations)? LR 

f. For changed functions, does the new logic 
mesh with the old? LO 


. Are all required data definition changes and 


additions specified and defined to the field 
description level (field names and storage 
attributes will be defined during detailed de- 
sign)? DA 

a. Is the field described correctly? 

b. Have any field definitions been omitted? 


. Are all required external linkage 


changes/additions specified and defined cor- 

rectly? LR | 

a. Should an external routine be used rather 
than performing the function internally? 

b. Does the processing module set up (for 
output) and process (on input) all required 
passed parameters? Correctly? 

c. Is all new/changed information flow across 
external linkages correctly described? Specif- 
ic return codes, parameter formats, etc., are 
not required; they will be documented in 
detailed design. 


. Are new/changed module attributes 


specified? MA 

a. Reenterable, reusable? 
b. Protection key? 

c. Fixed or pageable? 

d. Supervisor or problem? 
e. Enabled or disabled? 


. Are new/changed performance and storage 


criteria specified? PE 

a. Path lengths? 

b. Storage size? 

c. Is the function designed optimally for perfor- 
mance and storage? 


39 


d. Does the function use existing facilities (logic, 
control blocks, etc.) where possible? Cre- 
ation of new facilities should be avoided 
where existing ones are available. 

e. Will execution of the function as designed 
cause minimum (optimal) paging? 


Detailed Design Inspection Checklists 


Logic (LO) 

¢ Are all constants defined? 

e Are all unique values explicitly tested on input 
parameters? 

e Are values stored after they are calculated? 

«Are all defaults checked explicitly; for example, 
blanks in an input stream? 

e If character strings are created, are they com- 
plete? Are all delimiters shown? 

eIf a keyword has many values, are they all 
checked? 

«Are all keywords tested in a macro? 

e Are all keyword-related parameters tested in a 
service routine? 

e Are all increment counts S PrOpeny initialized 
(0 or 1)? 

«After processing a table entry, should any value 
be decremented or incremented? 

eIs provision made for possible processing at 
logical checkpoints in the program (end-of-file, 
end-of-volume, etc.)? 

els all 1/O performed on opened files? 

e Are routine error conditions adequately provided 
for (INVALID KEY, ON SIZE ERROR, etc.)? 

¢ Are literals shown where there should be constant 
data names? 

¢On comparison of group items, should all fields be 
compared? 

eIs the value of a data item used before the item is 
initialized? 

«Are all data areas shown in design necessary or 
are some extraneous? 


Data Area Usage (DA) 
eIf design is dependent on building/creating/ 
deleting various data areas, are all designated? 
¢ Should a called macro provide any INCLUDEs for 
any data areas that the macro expanded code may 
- depend on? 
e Does design show explicitly which area to use in a 
data area, that is, if there are multiple save areas? 
elf the program stores into a data area, does it 
store into the correct field? 
eIf a value is fetched from a data area, is the 
correct field fetched? 
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e Should the data area be boundary-aligned? 
e Does a Save area have multiple uses? Can con- 
flicts arise? 


~ Test and Branch (TB) 


e Are all three conditions tested, that is, greater 
than, equal to, and less than zero? 

e After a linkage, should a return code be tested? 

eIs a SORT or a MERGE operation tested for 
successful completion? 

e Are branch legs correct, that is, phous YES be NO 
and NO be YES? 


Return Codes/Messages (RM) 

e Are messages issued for all error conditions? 

eOn exits, should a return code be set or a message 
issued? 

e Does the message say what it means? 

e Could more information be supplied in the 
message? 

eDo return codes in the design for particular 
situations match the global definition of the 
return code as documented? 


Register Usage (RU) 
«If a specific register is required, is it specified? 
e Does any macro expansion use a register already 
in use without saving the data? 
els the integrity of all input registers maintained? 


More Detail (MD) 
¢ Does the design specify a process ambiguously, or 
does the process require more than ten instruc- 
tions? | 


External Linkages (EL) 
e Should a standard linkage be used rather than 
coding a subroutine inline? 
els the designated linkage the right one for the 
function to be performed? 
els the data area mapped as the receiving module 
expects it to be? 


Standards (ST) 
e Are any programming dander: for the project in 
jeopardy of compromise because of the design? 


Initial Design Documentation (HL) 
e Does the detailed design match the initial design? 
If not, the initial design documentation could be 
in error. 


Performance (PE) 
e Does the design impair the performance of this 
module to any significant degree? | 


Code Inspection Checklists 
COBOL Checklist 


Identification Division 
¢ Does the prose in the REMARKS paragraph 
- function as a complete prologue for the program? 
~PR 


Environment Division 
e Does each SELECT sentence explicitly define the 
external (system-dependent) specifications for the 
file? SU 


Data Division — File Section 
e Are the file definitions (FDs) in the same order as 
their respective SELECT sentences in the environ- 
ment division? DA 
e Do the record and data item names conform to 
their usage? DA 
e Does each FD contain comments regarding: DA 
File usage (recording mode), block size, record 
length, imbedded keys, etc.)? 
Amount of activity (updated how often, used 
every time program is run, etc.)? 
Interaction with other data items. (Do its 
records contain objects of OCCURS ... DEPEND- 
ING ON clauses? Is the length of its records 
dependent on such an object elsewhere in the 
program, etc.?) 
eIs the file sorted or merged? EL 
«Are statistics kept on file activity in a given run or 
series of runs? EL 
eIs the correct balance struck between specifying 
complete file attributes in the program and speci- 
fying some of them dynamically (such as block 
size, maximum record length); that is, if a file is 
designed to be flexible in the given program, is it 
defined as flexibly as needed? DA 


Data Division — Working Storage and Linkage 
Sections 
e Do the data item names conform to their usage? 
DA 
e Does each data item (except for elementary items 
of obvious usage — subscripts, etc.) contain 
comments regarding: DA : 
Characteristics (fixed- or variable-length, 
maximum allowable length, etc.)? 
Interaction with other data items? (Does this 
data item contain or depend on objects of 
OCCURS ... DEPENDING ON, etc.?) 
Area of use in program? (Is it used only ina 
certain section, or during a range of paragraphs, 
etc.?) 


¢ Are all data items with any kind of unifying 
quality placed together according to a particular 
scheme? DA 
Usage (arithmetic work areas, work areas for 
file records, etc.)? 
Commonality of purpose (everything used to 
process a particular file, etc.)? 
Attributes (message texts, constants, etc.)? 
e Are all working storage items that are used as 
constants designated as such? DA | 
e Are data items that are required to be in a partic- 
ular order sequenced correctly? DA 
els the use of REDEFINE/RENAME in a data 
description justified and documented in terms of a 
simplification of data references, rather than 
reliance on the normal hierarchy of level num- 
bers? SU 


Procedure Division 

e Are block comments included for major function- 
al areas (for example, paragraph, section, seg- 
ment)? CC 

eIs the module commented on in sufficient detail? 
Ce 

e Are comments accurate and meaningful? CC 

e Does the code essentially correspond to the » 
outline of the module documented in the remarks 
paragraph? LO 

e Does each paragraph, section, or segment have a 
homogeneous purpose which justifies and/or 
necessitates placing all the code together under 
such a grouping? MN 

¢ Does each performed paragraph or section 
document the function it accomplishes and the 
part of the overall logic it represents? CC 

eIn a segmented program, is it clear why segmenta- 
tion is necessary? MN 

¢ Does each segment stand alone, or is there heavy 
dependence on other segments? MN 


Format 

e Are IFTHENELSE and DO groups aligned proper- 
ly? MN 

e Are nested IFs indented properly? MN 

«Are comments accurate and meaningful? MN 

«Are meaningful labels used? MN | 

e Are the clauses of complex verbs (for example, 
SORT/MERGE and OPEN/CLOSE) indented 
properly and clearly under the verb? MN 

e Does all use of GO TO conform to installation 
standards? MN 


External Linkages 
e Are initial entry and final exit correct? EL 
els each entry point defined correctly? EL 
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eIs each parameter referenced in an ENTRY state- 
ment a 77 or O1 item in the linkage section? EL 
els the usage of STOP,RUN/GOBACK/EXIT 
PROGRAM verbs correct? EL | 
¢For each external call to another module: EL 
Are all required parameters passed to each 
called module? | 
Are the parameter values passed set correctly? 
Upon final exit from this module, are all files 
closed? 


Logic 

¢ Has all design been implemented? LO 

¢ Does the code do what the design specified? LO 

els the design correct and complete? LO 

e Are the proper number of characters within a 
field tested or set? LO 

eIs each loop executed and the correct number of 
times? LO 


Program Language Usage 

els the optimal verb or set of verbs used? PU 

eIs the installation-defined restricted subset of 
COBOL used throughout the module? PU 

eIs attention given to normal “‘housekeeping”’ 
requirements in COBOL (for example, setting the 
length of a variable-length target field before a 
MOVE to that field is executed)? PU 


Storage Usage 

eIs each field initialized properly before its first 
use? SU 

els the correct field specified? SU 

If a storage area is set and used recursively, is its 
housekeeping performed properly? SU 

els the field initialized statically (that is, by means 
of the VALUE clause on its definition), when it 
should be dynamically (by assignment), or vice 
versa? SU 

eIs the use of the REDEFINES clause in the data 
item’s definition compatible with all uses of the 
data item in the code? SU | 

e If the CORRESPONDING option of the MOVE and 
arithmetic verbs is used, is it absolutely clear from 
the data definitions which target fields will be 
affected (and, equally important, which 
will not)? SU,MN 
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Test and Branch 

els the correct condition tested (IF X=ON vs IF 
X=OFF)? TB 

eIs the correct variable used for the test (IF X=ON 
vs IF Y=ON)? TB 

els each condition name, used as a test of a data 
item, defined as an 88-level under that data item? 
TB 

els each branch target of a simple GO TO or GO TO 
... DEPENDING ON Statement, correct and exer- 
cised at least once? TB 

els the most frequently exercised test leg of an IF 
statement the THEN clause? TB 


Performance | 
els logic coded optimally (that is, in the fewest and 
most efficient statements)? PE 
e Has subscripting been used where indexing logic 
would be more effective and appropriate, or vice 
versa? PE 
e Have ERROR DECLARATIVEs been coded for files 
likely to have recoverable I/O errors? PE 
¢ Are normal error/exception routines provided 
for: PE 
ON SIZE ERROR — for arithmetic statements? 
INVALID KEY — for start/read/write/rewrite 
statements? | 
AT END — for search/release/sequential READ? 
ON OVERFLOW — for STRING/UNSTRING? 


Maintainability 

e Are listing controls utilized to enhance readability 
(for example, EJECT, SKIPx)? MN 

e Are paragraph and SECTION names consistent 
with the logical significance of the code? MN 

eIs each PERFORMed paragraph terminated with an 
EXIT paragraph? MN 

els the use of the ALTER statement completely 
justified, as opposed to some sort of 
switch/conditional branch flow of control? MN 

e Are null ELSEs included as appropriate? MN 


Copy Facility Usage 

eIs every data item definition and processing 
paragraph, standardized for the installation, 
generated in the module via the COPY facility? 
OT 

els there a sound reason why the REPLACE option 
of the COPY statement is utilized to change any of 
the names of data items in the COPY’d code? OT 


PL/I Checklist 


Code Documentation Prologue 
e Are there discrepancies between the code and the 
prologue? PR 


Procedure Statement (external procedure) 

eIs the procedure name the same as the module 
name? MN 

e Are the options present consistent with module 
design; that is, both the OPTIONS sublist 
(reentrant, etc.) and the options for the PROC 
statement (recursive, etc.)? OT 

e Are any required options missing? OT 


Initial DECLAREs and INCLUDEs, Storage 
DECLAREs 
«Do the names of the data items correspond to 
their usage? MN 
e Does each data item (except for items of obvious 
usage — constants, subscripts, etc.) contain com- 
ments regarding: CC 
Characteristics (fixed- or variable-length, 
maximum, allowable length, customary 
(arithmetic) precision, etc.)? 
Interaction with other data items? (Does this 
data item determine the length of another data 
item? Is this data item used to control a major 
loop, etc.?) 
Area of use in the program? (Is it used only in 
a certain section, or during a particular internal 
procedure, etc.?) 
e Are all related data items placed together accord- 
ing to a particular scheme? 
Usage (arithmetic work areas, work areas for 
file records, etc.)? MN | 
Commonality of purpose (everything used to 
process a particular file, arguments passed to 
the same subroutine, etc.)? MN 
Attributes (message texts, constants, etc.)? MN 
e Are all data items used as constants designated as 
such? MN 
e Are data items that are required to be in a partic- 
ular order sequenced correctly? MN 
e Do the INITIAL values conform to the 
design? LO 
e Are the correct data attributes used, that is, fixed, 
character, bit, decimal, etc.? DA 
els the correct storage attribute used (static, 
automatic, etc.)? DA 
e Are addresses defined correctly (use of BASED, 
ADDR function, etc.)? DA 
_ els the use of the DEFINED attribute in a data 
description justified by the simplification of data 
references? DA | | 


File DECLAREs 
eDo the names of the files, and the associated data 
items, correspond with their usage? MN 
e Does each file declaration contain comments 
regarding: CC 
File usage (record type, block size, record 
length, imbedded keys, etc.), provided such 
information is not clearly indicated elsewhere, 
such as the ENVIRONMENT attribute? 
Amount of activity (updated how often, updat- 
ed every time program is run, etc.)? 
Interaction with other data items? (Do its 
records contain data items which determine the 
length of other data items? Is the length of its 
records determined by a data item elsewhere in 
the program, etc.?) 
e Are statistics kept on file activity in a given run or 
series of runs? PE 
-eAre the correct file attributes used 
(STREAM/RECORD, SEQUENTIAL/DIRECT/ 
TRANSIENT, etc.)? OT 
els there a valid reason for placing file attributes 
on an OPEN statement, rather than in the file 
declaration itself? MN 
els the correct balance struck between specifying 
complete file attributes in the program, and 
specifying some of them dynamically at execution 
time (for instance, block size, maximum record 
length, etc.)’?) MN 


INCLUDEs 

e Are all data areas required by the module 
INCLUDEd correctly? DA 

els the INCLUDE feature (macro or option) used to 
define identical or nearly identical data areas 
(such as work areas and file areas)? DA 

eIs every data item definition and processing 
procedure standardized for the installation gener- 
ated in the module via the INCLUDE facility? OT 

eIs there a sound reason why any of the labels or 
names of the data items are changed in an 
INCLUDEd portion of text? OT 


Format | 

e Are block comments included for major function- 
al areas, such as the internal procedure 
section? CC 

e Are the module’s comments sufficiently 
detailed? CC 

e Are comments organized and formatted so that 
they do not interfere with the readability of the 
program? CC 

e Are comments accurate and meaningful? CC 
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¢ Does the code essentially correspond to the 
outline of the module documented in the pro- 
logue? PR : 

e Does each internal procedure or subroutine have 
a consistent purpose which justifies and/or 
necessitates placing all the code together under 
such a grouping? CC 

e Does each such internal. procedure or subroutine 
document the function it accomplishes and the 
part of the overall logic it represents? CC 

e Are meaningful labels used? MN 

e Are IFTHENELSE and do-groups aligned 
properly? MN 

e Are nested IF’s and do-groups indented 
properly? MN 

e Are block comments and remarks effectively 
positioned? MN 

e Are the clauses of complex statements indented 
properly under the verb? MN 

Complex arithmetic and assignment? 

File names and options on OPEN/CLOSE? 
ALLOCATE/FREE? 

CALL/ENTRY? 
FROM/INTO/KEYFROM/KEYTO 0n I/O sState- 
ments? 

Complex DO? 

GET/PUT? 

e Does all use of GO TO conform to installation 
standards? MN 


Entry and Exit Linkage (EL) 

e Are initial entry and final exit correct? 

els each parameter referenced in a PROCEDURE or 
ENTRY statement defined with the proper attri- 
bute (that is, compared to arguments in calling 
procedure)? 

e Are subroutines (internal PROCEDUREs) entered 
and exited properly? 

e Are options on internal PROCEDURE statements 
consistent with module design? 


Logic (LO) 

¢ Has all design been implemented? 

e Does the code do what the design specified; that 
is, is the design translated correctly? 

eIs the design correct and complete? 

e Are the appropriate number of characters within a 
field tested or set? 

els each loop executed the correct number of 
times? 


Program Language Usage (PU) 
els the optimal verb or set of verbs used? 
els the installation-defined restricted subset of PL/I 
used throughout the module? 
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e Are PL/I built-ins and functions used appropriate- 
ly in lieu of equivalent ordinary statements (for 
example, LENGTH, MAX, MIN, etc.)? 

els special attention paid to conversion rules? 

els an arithmetic built-in used instead of arithmetic 
assignment when difficult cases of precision arise 
(such as multiply, divide, etc.)? 

els attention given to normal “‘housekeeping”’ 
requirements in PL/I (such as setting the length of 
a variable-length target field before an assignment 
is made to that field)? 

e Are the proper scope rules of PL/I followed (for 
example, defining the same variable in procedure 
and in contained BEGIN block, etc.)? 

els the use of the LABEL variable completely 

- justified, as opposed to some sort of 
switch/conditional branch flow of control? 


Storage Usage (SU) 

e Is each field to be initialized set correctly? 

e Before the first use of any field, has it been 
initialized properly? 

els the correct field specified? 

e If storage is set and used recursively, is it 
‘“housekept” properly, depending on storage 
attributes (controlled, static, automatic), ona 
procedure basis, at least? 

els the field initialized statically (that is, by means 
of the INITIAL attribute in its DCL), when it 
should be initialized dynamically (by assignment), 
or vice versa? 

eIs the use of the DEFINED attribute in the data 
item’s definition compatible with all uses of the 
data item in the code? 

elf a BY NAME assignment is done, is it absolutely 
clear from the structure definitions which target 
fields will be affected (and, equally important, 
which will not)? 


Test and Branch (TB) | 

els the correct condition tested (IF X=ON vs IF 
X=OFF)? 

els the correct variable used for the test (IF X=ON 
vs IF Y=ON)? 

e Are null THENs/ELSEs included as appropriate? 

eIs each branch target correct and exercised at 
least once? 

eIs the most frequently exercised test leg of an IF 
statement the THEN clause? 


Performance (PE) | 
eIs logic coded optimally (that is, in the fewest and 
most efficient statements)? 
e Have ON statements been coded for files likely to 


have recoverable I/O errors (TRANSMIT, UNDE- 
FINEDFILE, KEY, etc.)? 

e For array assignments, would a more efficient 
assignment operation result from a subscripted 
do-loop, or vice versa? Similarly for structures? 

¢ Are normal error/exceptional conditions provided 
for via ON (condition) statements? 

ENDFILE/KEY/ERROR? 
CONVERSION/OVERFLOW/SIZE? 


Maintainability (MN) 
¢ Are listing controls utilized to enhance read- 
ability? 
% PAGE, %SKIP in macros/preprocessor? 
Control characters/blank cards in regular 
source? 
e Are labels and PROCEDURE names consistent 
with the logical significance of the code? 
e Are paired DO and END labels used? 
els proper formatting (such as indenting) provided 
either by the programmer or the compiler? 


External Linkages (EL) 


¢ For each linkage call to either a macro or another 
module: 
Are all required parameters passed to each 
linkage? 
Are the parameter values passed set correctly? 
If the linkage is a macro: 
Does the inline expansion contain all required 
code? 
Are there storage conflicts between macro and 
calling module? 
eIf the linkage returns, are all returned parameters 
processed correctly? 
¢ Upon final exit from this module, are all files 
closed and CONTROLLED storage FREEd? 


Indirect Addressing (OT) 
¢Is each pointer notation that is used correct? 
elf pointer/offset notation is used, is it required 
(that is, can a more direct method of addressing 
be used)? Similarly for the I-sub feature of 
arrays? 
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FORTRAN Checklist —_ 


Comments 
e Are comment lines used to group logically related 
statements? MN | 


Data Declaration 
e If COMMON contains many entries, check that 
repetitions in other subroutines match in length 
and order of listing. DA 
- elf EQUIVALENCE is used, check shared data 
storage for problems of unintended use. DA 
eCheck that FORMAT statements match READ, 
WRITE lists and that the intended conversion of 
‘data is specified. DA 


e Are FORMAT statements grouped at t the beginning 


of the program? MN 
e Are meaningful names used? MN 


Control - 

e Check nesting of DO ieee TB 

If extended range of DO is used, check 

carefully. LO 

_ eCheck whether any DO variable is to be used 
upon exit of the DO loop. LO 

_ «Does any use of GO TO conform to installation 
‘standards? LO | 


Coinpatsiini 
e Check all mixed mode expressions in assignment 
statements. DA 
e Examine all usage of complex numbers. LO 


| | Data Handling 


e Check that indexing out of the range of an array | 


does not inadvertently destroy constants or data 
areas. SU 
Format 
e Are installation indention standards . 
followed? MN 


Call Usage 


e Watch for the use of call by value, call by 
name. EL 
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e Match parameter list in caller-called program. PU 


Subroutine Usage | | 
e Check that any needed local variables have not 
been destroyed by consecutive invocations. LO 


Variable Types 
e Check that the correct variable types Gee 
real, complex, logical) are declared. PU 


Object Time Execution 

e Check job control language. OT 

e Check the use of associated variables in direct 
access statements. LO- 

e Check for the destruction of constant values upon 
return of a subroutine. EL 
For example 

CALL SUB(2.,x,y) 


SUBROUTINE SUB(a,b,c) 

A=4., — | 
RETURN | 

in which 2. is destroyed. 


FORTRAN Cotreniains | 
e Check the use of FORTRAN default sonventions: 
PU 


Examples: 
A. 1) X=Y**12 
Ay Rey 1, 


These produce different answers. In case 1, 
multiplication is performed, while in case 2, 
logical expression routines are used. 
B. 1) X=X+4+2 == I[=I*2 
2) M=X+2. J=]*2. 
In case 1, the constant 2 must be floated, 
_ while in case 2, the number must be fixed. 


Error Analysis 
e Check that arithmetic expressions will be evaluat- 
ed as intended (use of parentheses). PU 
¢ Check whether the proper length of precision has 
been selected for calculation and whether con- 
_ stants match in type. PU 


Assembler Checklist 


Register Usage (RU) 

_ eCheck that base registers defined by USINGs are 
all loaded at the appropriate time, that is, before 
first attempted use. 

¢ Check that all temporary base registers are 
dropped when no longer needed. 

e Check to ensure that base registers cannot be 
destroyed during execution, particularly via calls 
to subroutines or across CSECT boundaries. 

e Check that all intended entry points are defined 
by ENTRY statements. Use the External Symbol 
Dictionary to verify their external status. 


Program Language Usage(PU) 

e Check for operation code misspellings that will 
nevertheless be accepted by the assembler be- 
cause the misspelling is another valid assembler 
operation code for which the operands have the 
same format as the intended instruction. Exam- 
ples are: 


Instruction 
As Intended As Misspelled 
a. SR SUBTRACT S SUBTRACT 
REGISTER 
b. “LA LOAD ADDRESS  L LOAD 
STH STORE HALF- SH SUBTRACT 
WORD | HALFWORD 
d. SRL SHIFT RIGHT SLR SUBTRACT 
LOGICAL LOGICAL 
REGISTER 


a. Omitting the “‘R”’ in the instruction: SR 5,8 
will yield the valid instruction: S 5,8 

b. Omitting the “‘A” in the instruction: LA 5,8 
will yield the valid instruction: L 5,8 

c. Omitting the “‘T” in the instruction: 
STH 5, XYZNUM will yield the valid instruc- 
tion: : 
SH 5, XYZNUM 

d. The error in this case can go either way, that 
is, from SLR to SRL or from SRL to SLR. For 
example, both instructions are equally plausi- 
ble: SLR 5,8 and SRL 5,8. 

e Check that Load Multiple (LM) picks up the 
desired sequence of fullwords and that they are 
placed into the expected registers. 

e Ensure that CLI is not used when TM is really 
required, that is, check that bit-switches are not 
confused with byte-switches. 

e Check that register 2 has not been unwittingly 
destroyed by a TR (Translate) or TRT (Translate 
and Test) instruction. 


e Check that expressions representing lengths are 

specified correctly: 
For example, 

MVC 0 (LABEL2-LABELI1,R6),0 (R3) 

VS 

MVC 0(LABEL2-LABEL1+1,R6), 0 (R3) 

or 

VS 

MVC 0(LABEL2-LABELI-1), R6), 0 (R3) 

¢ Check that all possible cases of conditional 

assembly parameters are generating the code 
expected. An assembly should be produced for 
all major cases and the logic of each compared 
with a card image printout of the source state- 
ments. 

e Check that use of unconditional branches con- 
forms to installation standards. 

e Check that a save area exists, if required, and is 
set up according to the prevailing operating 
system conventions (for example, 
forward/backward pointers, etc.). If available, a 
system macro should be used to establish save 
area linkages (for example, the OS SAVE macro). 

e Check that register usage conforms to the prevail- 
ing standards applicable to the project, if any. If 
no special standards are in use, operating system 
standards should be applied (for example, for OS, 
R13 is the save area pointer; R14, the return 
address; R15, the entry point address; R1, the 
parameter list pointer; R10 and R11, parameter 
registers). 

e Check that EQUATEs are all meaningfully de- 
fined; in particular, check that register EQUATEs 
such as R5 EQU 5 are not redefined as a shortcut 
method of introducing changes, as would be the 
case if the foregoing example were changed to R5 
EQU 6 to free up register 5, assigning its current 
use to register 6. 

e Check system macro calls to ensure that keyword 
parameters are not specified as positional parame- 
ters, and vice versa. For macros accepting mixed 
format (both positional and keyword parameters), 
a keyword parameter written in positional form 
might be accepted as meaning something else than 
intended. 


Data Area Usage (DA) 

¢ Check that DSECTs correspond in format to the 
data which they represent. 

eIf modifications have been made to a data struc- 
ture, for example, addition of fields within the 
structure (control block), check that required 
alignments are still preserved. Use particular care 
in the case of control blocks iteratively generated 
via conditional assembly logic. (Even if the first 
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block is right, subsequent blocks may not start on Code Comments (CC) 


the same type of boundary, causing program e Check that instruction-level documentation adds 
failure only when operating on blocks other than meaning to the code, for example, in the instruc- 
the first.) 7 tion SR R5,R5 ZERO RS, the comment ZERO R5 

: : 7 adds nothing to the content of the instruction, 

Maintainability (MN) while SR R5,R5 ASSUME NO REQUESTS PENDING 
e Ensure that extended mnemonics are used when- does add meaning. | 

ever possible rather than hand-coded condition- 
code masks. . Logic (LO) 

| : eIs each loop executed the correct number of 

times? | 
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Appendix E: Reporting Forms 


Module Detail Reports 

The module detail report is completed for each 
module in which valid errors are discovered during 
an inspection. The following describes the use of 
each field of the three reports — initial design inspec- 
tion module detail report (Figure 22), detail design 
inspection module detail report (Figure 23), and 
code inspection module detail report (Figure 24): 


1. 


MODULE: The module name. 


2. COMPONENT/APPLICATION: The associated 


3. 


component/application name. 

PROBLEM TYPE: Summarize the number of 
problems by type (logic, etc.), by severity 
(major/minor — a major error is one that would 
cause the program to malfunction), and by 
category (missing, wrong, extra). For modified 
modules, problems in the changed portion and 


in the base program can be shown as follows: 
3(2), where 3 is the number of problems in the 
changed portion, and 2 is the number of prob- 
lems in the base. 


. REINSPECTION REQUIRED?: Indicate whether 


the module requires a reinspection. All valid 
errors found in the inspection are listed and 
attached to the report. A brief description of 
each problem, its error type, and the estimated 
rework time to correct it is included, as shown 
below. 


LO/M/IMAT THe VARY de ong 
Ctho-te. DEB a 
Wot | dia ( ptwo0k. = a ee 
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Figure 22. Initial design inspection module detail report 
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INITIAL DESIGN INSPECTION MODULE DETAIL REPORT 


Module: 


PROBLEM TYPE: 


LO: 


LR: 


DA: 


PE: 


MA: 


ES: 


SC: 


ST: 


OT: 


Logic 

Linkage Requirements 
Data Area Usage 
Performance or Storage 
Module Attributes 
External Specifications 
Specification Clarification 
Standards 


Other 


TOTAL 


Date 


Component/Application 


iE 


REINSPECTION REQUIRED?__(Y or N) 


DETAILED DESIGN INSPECTION MODULE DETAIL REPORT 


Date 


Module: Wu Component/Application 


PROBLEM TYPE: MAJOR | MINOR | — TOTAL 


co 

LO: Logic Z 
TB: Test and branch = 
DA: Data area usage = 
RM: Return codes/messages & 
RU: Register usage x 
EL: External linkages Z 
MD: More detail - 
ST: Standards a 
PR: Prologue or prose = 
HL: Initial design documentation a 
US: User specifications g 
MN: Maintainability _ 
PE: Performance a 
be 


OT: Other 


TOTAL | 


REINSPECTION REQUIRED? __ (Y or N) 


Figure 23. Detailed design inspection module detail report 


CODE INSPECTION MODULE DETAIL REPORT 


Module: 


LO: Logic 

TB: Test and Branch 
EL: External Linkages 
RU: Register Usage 
SU: Storage Usage 
DA: Data Area Usage 
PU: Program Language Usage 
PE: Performance 

MN: Maintainability 
DE: Design Error 

PR: Prologue 

CC: Code Comments 


| OT: Other 


TOTAL 


REINSPECTION REQUIRED?______ (Yor N) 


Figure 24. Code inspection module detail report 


a2 


Date 


Component/Application 


> 
i 
) 
es) 
= 
= 
) 
ss) 


TOTAL 


ba 
a 


pt ttt tt de 
ett tte tt te yt tt ts 


Summary Inspection Report 
The summary inspection report (see Figure 25) 
contains the results of the inspections of several 
modules (usually those forming a component of an 
application) and is distributed to design and develop- 
ment management. The following describes how 
each section of the report is used. 
REPORT NAME: The box is checked correspond- 
ing to the type of inspections summarized. 
SUBJECT: The unit inspected is identified. 
MODULE NAME: The name of each module as it 
resides on the source library. 
NEW OR MOD.: ‘‘N” if the module is new; ‘“‘M”’ if 
the module is ““modified’’. 
FULL OR PART INSP.: For a modified module, “‘F”’ 
if the module was fully inspected; “‘P”’ if the 
module was partially inspected. 
WORK PERFORMED By: The correct categories 
are checked and the individuals’ names specified. 
EST. PRE. ELOC/NCSS: ELOC is the estimated 
executable lines of source code made before a 
design inspection by the designer. NCSS is a count 
of the lines of noncommentary source statements 
made before a code inspection by the programmer. 
EST. POST. ELOC/NCSS: The estimate or count 
made after the inspection. 
REWORK ELOC/NCSS: The estimated executable 
lines of source code in rework as a result of the 
inspection. 


OVERVIEW AND PREP.: The number of person 
hours (in tenths of hours) actually spent preparing 
for the overview, in the overview meeting itself, 
and preparing for the inspection meeting. 

INSP. MEETG.: The number of person hours 
actually spent on the inspection meeting. 
REWORK: The estimated number of people hours 
spent to correct the problems found during the 
inspection. 

FOLLOW-UP: The estimated number of people 
hours spent by the moderator (and others, if 
necessary) in verifying the correctness of changes 
made by the author as a result of the inspection. 
COMPONENT: The component of which the 
module is a part. 

REINSPECTION REQUIRED?: Yes or no. 

LENGTH OF INSPECTION: Clock hours spent in 
the inspection meeting. 

REINSPECTION BY (DATE): Latest acceptable date 
for reinspection. 

ADDITIONAL MODULES?: For these components, 
are additional modules yet to be inspected? 

DCR’s ID’s WRITTEN: The identification of design 
change requests written to cover problems in 
rework. 

PROBLEM SUMMARY: Totals taken from module 


detail form(s). 


INITIAL DESIGNER DETAILED DESIGNER, etc.: 
Identification of members of the inspection team. 
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SUMMARY INSPECTION REPORT _ INITIAL DESIGN(__] DETAILED DESIGN {J CODE (_) 


Date 

To: Design manager Development manager 
Subject: Inspection report for Inspection date 

Application 

Component(s) 

Work Performed By 
Initial Detailed Inspection Person-Hours (X. an 
Full | Designer (CoJ| Designer CJ ELOC/NCSS | Estimated | 


New or Detailed iat Added, Modified, Deleted 
Module} or Part | Designer Cy} Est.Pre. | Est. Post. Rework 


Reinspection required? Length of inspection (clock hours and tenths) 
Reinspection by (date) Additional modules 


DCR ID’s written 


Problem summary: Major Minor Total 
Errors in changed code: Major Minor Errors in base code: Major Minor 
Initial Designer Detailed Designer Programmer Team Leader Other Moderator’s Signature 


Figure 25. Summary inspection report 
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- Appendix F: Test Plan and Test Case Inspections 


This appendix discusses two additional types of 
inspections — test plan and test case — as shown in 
Figure 26. Since the inspection process is basically 
the same as for other inspections, this appendix 
discusses in detail only those items that differ from 
the other inspection types. 

Test plan and test case inspections are designed to 
inspect the plan made for testing new and/or 
changed functions of a component and to inspect the 
test cases developed from that plan with the objec- 
tives of ensuring more complete and smoothly 
functioning testing with fewer testing resources. An 
indication of the productivity possibilities of test 
plan and test case inspections was provided by an 
IBM test in which four new functions approximating 
20K lines of source code were subjected to these 
inspection types in addition to the design and code 
inspections. It was estimated that personnel hours 
spent in testing and in test plan and test case inspec- 
tions were 85% less than the personnel hours that 
would have been spent in function testing without 
the test plan and test case inspections. (Note that 
these test plan and test case inspection results 
involved only one study. Since the results depend on 
many factors, they cannot be considered representa- 
tive of every situation. ) 


Test Plan Inspections 


Objectives 

The primary objectives of a test plan inspection are 
to verify that function testing will provide assurance 
that the function operates correctly within its intend- 
ed environment and the previously tested related 
functions still execute properly. A major part of a 
test plan is the identification and description of each 
test case. 


Process and Participants 

The test plan inspection process consists of the same 
steps as other inspections: planning, preparation, 
inspection meeting, rework, and follow-up. 

In addition to the moderator, the participants 

include: 

e Functional tester — the author of the test plan. 

e Functional designer — the designer of the function 
to be tested. The designer is the key inspector 
because it is assumed that a function’s designer 
has the most knowledge of the function being 
tested and can verify that the function will be 


comprehensively tested in an appropriate environment. 


Initial 


design 


Initial 
design 
inspection 


Detailed 
design 


Test plan 
preparation 


Detailed 
design 
inspection 


= 


Code 
inspection 


Unit 
test 
Function 
test 
System 
test 


Figure 26. Inspection types 


Test 
plan 
inspection 


Test case 
preparation. 


Test case 
inspection 
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e Other inspectors — these inspectors, from design 
and development groups, are usually responsible 
for incorporating the function into a particular 
program or application and can verify that the 
significant coding changes made to their modules 
(particularly linkages) will be tested. Even if one 
test plan includes several functions, there may be 
one inspection for each function. In such cases, it 
is worthwhile for one of the participants to attend 
all the inspections to provide continuity. Fre- 
quently, test cases designed to test one function 
can be slightly modified to test others. 


Preparation 

Each participant examines the test plan and the 
initial design documents to verify that the function 
will be properly tested. It is assumed that the test 
plan, at a minimum, includes the following 
information: | 

eA general test philosophy or strategy 

eA description of the function to be tested 

eA representation of functional coverage (that is, 
matrix, cause and effect graph, family tree, etc.) 

e A description of the conditions each test case will 
test and how this will be accomplished 

e Testing dependencies (hardware/simulator 
needs, etc.) 

e Entrance and exit criteria 

Participants in the inspection meeting review the 

planned testing activity as documented in the test 
plan and compare the plan to the initial design 
materials with the objective of trying to find errors 

_ in the test plan. As with other inspection types, a 
checklist can assist in focusing attention on possible 
errors. The checklist below is a sample of the items 
that could be included; it should be modified to meet 
the needs and standards of each installation. 

eIs the description of the function being tested, as 
documented in the test plan, complete and 
accurate? 

e Are the mainline and alternate paths listed 
sufficiently to provide confidence that the func- 
tion being tested operates correctly? 

els the testing approach feasible? 

¢ Are all the new and/or changed user linkages 
exercised? 

els a sufficient number of defaults exercised? 

e Are messages verified? 

e Are error paths exercised? 

- Are return codes generated? 

¢ Are sufficient and proper tests identified to 
reverify previously tested related functions 
(regression test cases)? 
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¢ Are there simulator and hardware dependencies 
that are not addressed? 

¢ Are test plan entrance and exit criteria realistic? 

«Are there any outstanding design changes to be 
made that will invalidate the completeness of the 
test plan? 


Rework and Follow-up 

The same considerations apply to test plan inspec- 
tion steps as to the corresponding steps of other 
inspection types. 


Reporting Forms 

Test plan inspection detail and summary report 
forms as shown in Figures 27 and 28. It is expected 
that installations will tailor them to their own needs. 


Test Case Inspections 


Objectives 
The primary objectives of a test case inspection are 
to review the test cases, now in executable form, to 
verify that: 
¢ The test cases cause those conditions specified in 
the test plan to be executed. 
¢ Each test case prologue provides a complete and 
accurate description of its purpose and expected 
results, and explicit instructions for its execution. 


Process and Participants 

The test case inspection process consists of the same 
steps as other inspections: planning, preparation, 
inspection meeting, rework, and follow-up. 

In addition to the moderator, the participants 

include: 

«The person responsible for the test cases being 
inspected. 

« An inspector experienced in running test cases. 
This inspector can determine whether the pro- 
logue identifies test case dependencies and pro- 
vides all the information needed by the operator. 

«An inspector experienced in developing test cases 
for this application. 


Preparation 
Each participant examines the test case data, com- 
paring each test case with its description in the test 
plan. The test case inspection materials include: — 
1. Test case data | 
2. Test case prologue containing the following 
information: 
¢ Description and purpose of the test case 
¢ Setup requirements | 


TEST PLAN INSPECTION FUNCTION DETAIL REPORT 


FUNCTION 


PROBLEM TYPE: 


FD: Functional description 
TP: Test procedure 

TS: Test strategy 

FM: Family tree/matrix 
TD: Test case dccee ation 
RT: Regression tests 

BR: Build requirements 
SH: Simulator/hardware 


OT: Other 


Subtotal: 


TOTAL: 


M = Missing, W = Wrong, E = Extra 
Moderator Tester 
Inspectors 
Reinspection required? Yes 
Person-hours expended: 

Planning ____ «Preparation 


Figure 27. Test plan inspection function detail report 


Date 


TOTAL 


= 
> 
cw 
‘e) 
Pe) 


* Typos, editorial changes, etc. 


Designer 


No SOs Estimated rework hours 


Inspection meeting 
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TEST PLAN / TEST CASE INSPECTION SUMMARY REPORT 

To: | Date: 

A ___. inspection meeting was held on 
System/application affected 
Function 

Results of the inspection are summarized below: 
Inspection meeting length | 
Number of problems found: 


Major SC Minor Total 


Reinspection required? Yes No 


Total estimated rework hours 
Total estimated follow-up hours 


Total test case data elements inspected 


Person-hours expended: 
Planning 
Preparation 
Inspection meeting 


Function or test case details are attached 


(Signed) 


Figure 28. Test plan/test case summary inspection report 
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e Operator instructions for running the test 
case 
e Normal and abnormal completion messages 
e Dependencies required by this test case, such 
as simulator, hardware, or test macros 
e Name of owner of the test case 
3. Sections of the test plan necessary to define this 
test case 
4. A copy of the initial design documents 
Participants in the inspection meeting review the 
test cases with the objective of finding errors in 
them. As with other inspection types, a checklist 
can assist in focusing attention on possible errors. 
The checklist below is a sample of the items that 
could be included; it should be modified to meet the 
needs and standards of each installation. 
Prologue: 
els the description of the purpose of this test case 
complete and accurate? 
e Are the operator instructions explicit and clear to 
ease test case execution? 
e Are all dependencies identified? 
e Are all normal and abnormal completion mes- 
sages identified? 
e Are setup requirements explicit and complete? 
e Is the owner of the test case identified? 
e Are there “‘progress’”’ messages identified that will 
notify the operator when significant parts of the 
test case are being executed? 


Test Data: 

¢ Does the test data follow its description as stated 
in the test plan? 

¢ Are the appropriate conditions established to test 
the intended variations? 

els the test for successful completion correct? 

¢ Are initial declares, respecify, and includes 
complete and correct? 

e Are entry and exit linkages correct? 

e Are macros issued properly? 

¢ Are appropriate return and feedback codes 
properly verified? 

e Are the messages identified in the prologue issued 
during the test case? 


Rework and Follow-up 

The same considerations apply to test case inspec- 
tion steps as to the corresponding steps of other 
inspection types. 


Reporting Forms 

The test case inspection detail report form is shown 
in Figure 29. The summary form is the same as that 
used for test plan inspections (Figure 28). It is 
expected that installations will tailor the forms to 
their own needs. 
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TEST CASE INSPECTION DETAIL REPORT 


Test Case 
PROBLEM TYPE: 
DR: Description 
Ol: Operator Instructions 
| MG: Messages 
DP: Dependencies 
LO: Logic 
RU: Register Usage 
MU: Macro Usage 
DA: Data Definitions 
LK: Linkage 
OT: Other | 
Subtotal: 
TOTAL: 


M = Missing, W = Wrong, E = Extra 


Date 


Function 


MAJOR* MINOR TOTAL 
pmMiwileim(wie} 


* Major = Problems that result in erroneous execution. of test cases 


Moderator 


Inspectors 


Reinspection required? Yes 


Number of data elements in test case 


Person-hours expended: 


Planning _s«~Preparation 


Figure 29. Test case inspection detail report 
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Tester 


Estimated rework hours 


Inspection meeting 
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Added August 15, 1978 
By TNL GN20-3814 


Appendix G: A Report on the Use of Inspections for an Applications 


Development Project within IBM 


This report describes the use of inspections in an 
applications development project in IBM’s Informa- 
tion Services Ltd., England and was prepared by 
individuals in close contact with the project. The 
objectives of the pilot project were to check as to 
whether inspections did increase productivity and 
program quality. Quality was to be measured by the 
APAR (reportable error rate), with an objective of 
one valid APAR per thousand lines of code in the 
first year of production. As a more immediate 
indication of quality, the system test error rates 
would be compared with a project that used struc- 
tured walk-throughs. Since productivity increases 
are much more difficult to measure because of the 
lack of an accurate estimating method, this area was 
to be treated more subjectively. 

Another objective was to determine the accept- 
ability of the process to the team members and the 
project managers, with emphasis on the impact on 
the workload and the schedule; and also to judge the 
effect on the individual of the open examination of 
an individual’s work, with the consequent disclosure 
of errors. 


ProjectDescription 

The inspections pilot project was the IBM World 
Trade Parts Returns System (PRS), a system to track 
and report on parts taken out or returned by CE’s. 
PRS 1s IMS-based, primarily data base, but with one 
data communications component. The original 
estimate was 5000 lines of PL/I code in nine sepa- 
rately identified programs. The final count was 
6271 lines of code. 

The team used most of the improved program- 
ming technologies (structured programming, top- 
down development, development support libraries, 
Hierarchy plus Input-Process-Output (HIPO), and 
chief programmer teams) during system develop- 
ment. However, they used inspections rather than 
walk-throughs, used top-down design starting with 
the detailed design level, and documented the design 
with a combination of HIPO charts and prose de- 
scription. Although structured programming tech- 
niques were used, strict structuring was not always 
apparent. TSO 3270 Display Support and Structured 
Programming Facility (TSO/SPF), program product 
5740-XT8, was used for library management. 

The team consisted of five people — a project 
leader and four programmer/analysts, one of whom 
was the chief programmer. The project leader 
devoted the greater part of the time to project 


control and took only an indirect part in the design 
and coding of the system. However, the leader 
made a point of attending almost all of the inspec- 
tion meetings. The rest of the design/code responsi- 
bilities were shared by the four programmer/ 
analysts. Average DP experience of team members 
at the beginning of the project was about 4 1/2 
years, with a minimum of 2 1/2 years and a maxi- 
mum of 6 1/2. 


ImplementationApproach 

To allay the fears that individual performance 
would be judged on the basis of inspection error 
lists, it was decided that all statistics would be kept 
within the group for the first month, at which point 
a meeting of all the interested parties would be held 
to resolve issues, such as use of statistics, and to 
review progress. It was agreed that at any sign of a 
major adverse effect on team morale inspections 
would be discontinued. 

Before any inspections started, a presentation 
describing the process and its objectives was given to 
the team and its managers. Shortly afterward, lists 
of exit criteria and error checklists for the detailed 
design and code inspections were distributed to all 
team members. It was decided to implement inspec- 
tions at the current position without any backtrack- 
ing. This meant that inspections for overviews of 
system components were not held, nor were some 
initial design inspections. 

It was apparent early that a flexible approach 
would have to be adopted, because the ground rules 
laid down by Fagan* were derived from a systems 
software environment, an environment that differs 
markedly in some respects from the applications 
environment. An example of the sort of adjustments 
that had to be made was the transfer of test plan 
inspections to code inspection time because it was 
found that examining a series of test cases with 
supporting data was more easily done in parallel 
with the inspection of logic. 


*M. E. Fagan, “Design and Code Inspections to Reduce Errors in 
Program Development”, IBM Systems Journal, Volume 15, 
Number 3, 1976. Reprints are available from IBM under order 
number G321-5033. 
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Later, it was decided to inspect test situations 
(without the data) at detailed design inspection time 
as well when the situations and the design specs 
acted as a mutual check on the completeness of one 
another. This approach proved successful and was 
adopted for the rest of the system. 

The selection of attendees at the inspections was 
the responsibility of the team leader. The whole 
team attended the first design inspection for each 
program for education purposes: an initial design. 
inspection where possible, or where this had been 
missed, the detailed design inspection. Thereafter, 
usually three people from the team attended — 
typically, designer, coder, and team leader, with the 
moderator bringing the number to four. However, 
this arrangement was changed whenever necessary; 
for instance, for the most important part of the 
system, or where an interface involving more people 
was being inspected. No problems with the size of 
inspection meetings were encountered, although six 
people should be an upper limit while four is an 
ideal number, promoting maximum interaction with 
minimum communication problems. 

Scheduling of inspections was also done by 
negotiation with the team leader, who was in the 
best position to know when work would be ready. 
When a program required several inspections, major 
branches of the hierarchical structure were chosen as 
material for each meeting, at first using the working 
rates given as guides by Fagan and later drawing 
upon experience. The moderator usually arranged 
for the meeting room. 

The conduct of the inspections quickly settled 
down into a routine with all participants knowing 
what was expected of them. The meeting would 
start with the moderator recording the times spent 
by individuals in preparation, and then the reader 
would start paraphrasing the material to the rest of 
the team, while they followed from the documenta- 
tion. As errors came to light, the moderator record- 
ed these briefly, while the rest of the team proceed- 
ed. These brief notes were read back at the end of 
the meeting for the agreement of the team, and were 
expanded and classified on a formal error list 
afterwards. It took several meetings for the mechan- 
ics of the error discovery process to become familiar 
to the team members; but once learned, the meetings 
proceeded more quickly and smoothly, and it was 
noticeable that the lessons learned from the error 
discovery process were being applied as new materi- 
al was submitted — in fact, a learning process by 
feedback was taking place. 

Sometimes team members would, in inspection 
meetings, disagree with various aspects of the 
project. The resulting discussion either clarified the 
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situation for everyone or showed that further inves- 
tigation was necessary. However, not all such 
discussions were as immediately productive, because 
some time was wasted over differing interpretations 
of standards. The project standards manual, while 
perfectly adequate for the normal working environ- 
ment, was still not as complete or as formally 
defined as necessary for development using the 
inspections process. 

The team in general accepted inspections quite 
readily, although accepted work practices were 
somewhat changed by it. What this meant in 
practice was that peak project loading was moved 
from the end of the development phase to the early 
part. 

On a personal level, the inspections environment 
demands an attitude to one’s work radically differ- 
ent from that of the traditional programmer. Where- 
as programmers have in the past been secretive 
about their work and defensive of their style, inspec- 
tions expose their work to the critical scrutiny of 
their colleagues. Although one person on the 
project reacted defensively to the identification of 
errors in that person’s work, not only did this team 
member’s work come successfully through the 
inspection process but the individual became more 
comfortable with the process. 

‘This experience demonstrates that while not 
everybody is necessarily ideally suited by tempera- 
ment to the inspection process, this need not pre- 
clude the use of the technique. Tact and patience 1s 
required of the team, whoever’s work is being 
inspected, and it is probably true to say that in- 
creased team spirit and cooperation was the result in 
this group. 


Results 

The results of the pilot project will be discussed in 
relation to the stated objectives: to verify the 
benefits, and the acceptability of inspections to the 
project personnel. In addition, some statistics will 
be presented that quantify basic parameters of the 
inspection process in the project environment, and 
could be useful to anyone implementing inspections 
for the first time. The benefits to be verified were 
quality and productivity, and increased profession- 
alism of team members. Quality was further subdi- 
vided into reliability, modifiability, and predictabili- 
ty (a measure of how the project meets its budget, its 
schedule, and its expectations). 


Professionalism 

Within the inspections environment there is a 
positive feedback mechanism for both authors and 
inspectors. To demonstrate this, errors for each 


designer and programmer, found during detailed 
design and code inspections and normalized to error 
rates per thousand lines of code (errors/KLOC), were 
plotted against the calendar date of the relevant 
meeting (see Figure Gl). Then curves were drawn 
through the plotted design and code points for each 
individual. Unconnected points on Figure G1 
represent the error rate for individuals who submit- 
ted only one piece of design or code. 

The curves shown represent a downward trend in 
the error rate made in both design and coding 
operations. Because a uniform postinspection 
standard was achieved for all programs, this trend 
represents an increase in technical competence of 
the individual; an increased awareness of the errors 
one 1s prone to make results 1n a positive improve- 
ment in programming efficiency. No attempt is 
made to quantify this increase because of the 
varying complexities of programs and experience of 
programmers, but the trend shown over a period of 
six weeks during the coding phase is very encourag- 
ing for such a short period, and suggests that inspec- 
tions provide a powerful learning mechanism. 

This effect is apparently not confined to authors, 
because individuals who submitted only one piece of 
work near the end of the coding phase (represented 
by the unconnected points on the graph) had results 
that followed the trend indicated by the curves, 
which suggests that by acting as inspectors they 
benefited vicariously from the inspection results of 
those who had gone before. 

When this subject was discussed with the team 
members, they all felt they had benefited from the 
learning mechanism, but the more experienced 


30 


ERRORS/ 
KLOC 


1 1] 21 311 11 25 
JANUARY FEBRUARY 


Page of GC20-2000-0 
Added August 15, 1978 
By TNL GN20-3814 


people less than the others. Some people also felt 
that the discipline inherent in the inspections process 
forced them to concentrate more on detail checking, 
to the benefit of their work. All team members felt 
that better communications throughout the team 
were a very real benefit, with everyone conversant 
with all parts of the system. This means that backup 
and maintenance problems are eased. 

On the more personal level of job satisfaction 
derived from this method of working, it was appar- 
ent that the involvement in other inspections meant 
more variety of work. 


Reliability 

Reliability in software is a deceptively simple 
quality that is not only a function of the number of 
errors found in system tests, or in the field, but also 
has connotations of trust and confidence. Thus any 
discussion of reliability should not be confined to 
error statistics, but should also include the more 
personal or abstract considerations of those respon- 
sible for the work. A measure of reliability, there- 
fore, results not only from the removal of errors, but 
from a feeling of assurance that all errors have been 
removed. 

Figure G2 shows the error rates experienced, and 
the hours of effort required to find them. It is 
interesting to note that during the design phase the 
majority of errors are of the “missing” category, 
while the majority made during coding are in the 
“wrong” category. It is encouraging that missing 
function is detected at the design stage, because 
missing function is traditionally the most difficult to 
fix after coding is complete. 


x x CODE 


e—___——e DESIGN 


MARCH APRIL 


Figure G1. Individual error rates during development for designers and programmers 
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Initial design 
Detailed design 
Code 


O| 2 5 | 11 
28 | 11 82 | 53 
40 | 50 39 | 99 


M = Missing 
W = Wrong 
E = Extra 


Error Classification: 


Figure G2. Error rates and detection effort 


The greater error rate at the code stage is probably 
due to the differing characteristics of the design and 
code targets. Design is read by a person, while code 
is read by a compiler — the compiler has no imagina- 
tion while a person can guess intent. The human 
ability to interpolate, combined with the expressive- 
ness and flexibility of the English language, means 
that detail and function are fairly easily omitted 
from the design. On the other hand, the inflexibility 
of a compiler and its rigid syntax rules mean that it 
is much easier to make a mistake in expressing an 
intent. 

Consistency in reliability is an important factor, 
on the principle that a chain is only as strong as its 
weakest link. The inspections process aims for 
uniformly high quality. To test the uniformity of 
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the quality, the number of errors found in each 
program during unit/integration test was recorded 
and plotted on a graph against time. The plot was of 
an error rate, normalized to errors per thousand 
lines of code (KLOC), against the calendar date of the 
last code inspection for that program. The results 
are shown in Figure G3, with a “least squares fit” 
relation plotted. The slight downward trend of this 
curve indicates that no more errors were detected 
during the later stages of unit test than in the earlier 
stages. The conclusion drawn from this is that, 
similarly, no more errors remained at the end of the 
inspection process for the later programs than the 
earlier ones and that, at least, the process was 
consistent in the quality of output, and possibly with 
a slight increase in quality with time. 


E=7.7-0.023T 
(Line of least squares fit) 


MARCH APRIL 


Figure G3. Errors remaining after code inspection and found during unit test 
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The team felt a little doubtful that all parts of the 
system would prove equally reliable, but believed 
they knew where problems might arise. This was a 
reaction based on their knowledge of the complexi- 
ties of the programs. 


Modifiability and Predictability 
The ease with which program function can be 
changed or added to is the quality of modifiability 
and is largely a subjective matter (though if the 
principles of structured design* are strictly followed, 
it is possible to derive a quantitative measure). Very 
few changes involving more than a line or two of 
code were made at any time during development, 
but where this had to be done the programmers 
involved found that the affected areas were easily 
isolated, and the changes could be quickly made. 
Although this was not a direct consequence of the 
inspection process, it was one of the aims of the 
detailed design inspection meetings to achieve a 
“sood” design. Where this objective could be said 
to have been missed was in the size of modules: in 
several cases, modules of up to 200 statements were 
recorded. The contrast between those and the more 
usual one-page modules was very evident during 
code inspection meetings and provided convincing 
support for the “small module” argument. 
Predictability, like modifiability, is largely subjec- 
tive. The managers directly involved were worried 
by the steadily rising overtime figures in the middle 
of the development phase. However, during the 
latter half of detailed design, the overtime figures 
steadily declined to zero (instead of, as is usual, 
rising to a peak at system test). The hump was due 
to a transition from design to code (when both 
activities were being carried out concurrently) and 
the absence of the moderator for a week, which 
tended to compress activities on either side. The 
programs all met systems test start date, and were 
fully unit/integration tested (except for one program 
not on the critical path) in an unstressed atmo- 
sphere, so the project can be said to have met its 
schedule. The manual project tracking system used 
by the team leader employed exits from inspections 
as milestones. 


Comparison with Another Project 

A similar project was selected for the purpose of 
comparing reliability by means of system test results. 
The project (project X) used improved programming 
technology techniques throughout, apart from the 
chief programmer team concept. Except that project 
X used walk-throughs instead of inspections, imple- 
‘mentation was very similar to PRS. As can be seen 
from Figure G4, which summarizes the two projects, 
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there is a significant improvement in the test error 
rate for the project using inspections, while the 
coding rate is similar. The coding rate is for the 
total development process (design, code, and test), 
and the PRS figure includes the moderator’s time, 
approximately 10%. It would appear that an error 
frequency of only 20% of that of previous products 
can be expected for no increase in manpower. 
Project X is a highly regarded system, remarkable 
for rigorous development and testing. The fact that 
PRS was able to improve on its quality is an indica- 
tion of the effectiveness of the inspections 
technique. 


PROJECT X PRS 


Improved Programming Yes Yes 
Technologies 


Reviews Walk-throughs Inspections 
Number of statements 10,000 6,250 
Total detail design, 

code, and test personnel 64 person 41 person 


months months 


Duration 14 months 7 months 


System test errors 51 11 
Pilot installation 26 O (also O errors 


errors in first 6 months 
of operation) 


Coding rate (LOC/ 
person months) 


Test error rate 
(errors/K LOC) 


Figure G4. Summarizing project X and PRS 


Working Rates 

Figure G5 shows the working rates achieved by the 
PRS team for the two basic types of inspections: 
detailed design and code inspections. As will be 
seen, code inspections proceeded much more slowly 
than design, especially in preparation. This possibly 
reflects the relatively large size of some modules; it 1s 
likely that rates would improve if module size were 
restricted to a page of printout. 


*W. P. Stevens, G. J. Myers, and L. C. Constantine, “Structured 
Design”, IBM Systems Journal, Volume 13, Number 2, 1974. 
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Inspection 


Preparation 


Design 275 275 


Code 


125° 


Figure G5. Working rates experienced by PRS team during 
detailed design and code inspections 


Productivity 

Although the moderator represents an additional 
overhead to the system development process, 
probably in the order of 10% in a fully established 
inspections environment, Fagan claims that there 1s 
a net productivity gain, and this seems to be borne 
out by the PRS project experience. However, it is 
likely that the ultimate justification will be in the 
reduced maintenance overhead from shipping a 
higher quality product. In practical terms, this 
means that making the effort earlier makes things 
easier later. 

The following is an estimate of the productivity 
gain derived from the use of the inspections tech- 
nique in the PRS project. It will be seen that much of 
the gain resulted from errors being detected in an 
earlier stage of development rather than during unit, 
system, or field test. The savings estimates are based 
on the following assumptions: 

1. PRS will be released to the field with one valid 
APAR per 1000 lines of code (a total of 6 APARS 
since program size is approximately 6000 lines 
of code). 

2. To compare the errors found in the various PRS 
inspections with errors in a project in which 
inspections were not used, assume that the 
errors in the noninspections project would be 
found in the unit, system, and field test in the 
same proportion as they were found during the 
PRS project. With this assumption, the error 
figures break down to: 


Implementation In- Unit Sys Fld. Tot. 
spec- test test test 
tion 
Using 
Inspections 
Without 


136 41 Il 6 194 


317 AD 194 


Inspections - 138 
*This figure agrees fairly well with what would be expected 


from project X experience (see Figure G4). The project X 
figures scaled to PRS size would lead one to expect in the 
order of 32 system test errors. 
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3. Approximate average error correction times are: 
e Inspection errors (I) — 1 hour 
e Unit test errors (U) — 3 hours 
e System test errors (S) — 7 hours 
e Field test errors (F) — 20 hours 
If the average time to fix errors at each stage are I, 
U, S, F respectively, then the total saving due to 
iMspections is: 


138U + 37S + 19F 
— 41U + 11S + O6F + 136! 
97U + 26S + 13F - 136! 


97(3) + 26(7) + 13(20) - 136(1) 
Thus, the total estimated saving is 597 person- 
hours, or 17 person-weeks, assuming a 36-hour work 
week. Since the project required 41 person-months 
overall, this 17 person-weeks saving translates into a 
net savings of 9% due to inspections. 


Conclusions 
The disciplined team approach to verifying quality 
brought about by inspections has resulted in the 
following benefits for the PRS team. 
eA high standard of quality in the finished prod- 
uct that compares favorably with comparable 
projects using improved programming technology 
techniques including walk-throughs. 
eA satisfying working environment for team 
members. 
eA net productivity rate comparable to similar 
projects. The overhead (10% maximum) added 
by the moderator is more than compensated for 
either by increased productivity or by reduced 
maintenance. 
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AdditionalInformation 

Figures G6, G7, and G8 show code inspection and 
design inspection criteria used by the project as well 
as a sample inspection error list. 


Module prologue must be complete and up to date. 
Code must be structured, and formatted to reflect the structure. 


Code must be sufficiently commented upon. 


All project standards and conventions must be followed. The compiie must be error-free, 
with no messages with a severity level greater than ‘‘warning’’. 


The listing should include cross-reference listing and aggregate list. 


The code must reflect the current design specification. 


Figure G6. Code inspection exit criteria 


1. Completed specifications must represent design to the approximate range of 3 - 10 source 
language lines of code per design process statement. 


2. Design specifications must be structured (all applicable structured programming rules are 
to be followed). 


3. For design specifications covering modified modules, all new/changed statements should 
be flagged with release identification. 


4. References to data areas should be by field name. Specific values tested or set should be 
specified. 


5. Linkages to external routines/modules must be defined. Parameter values passed and 
return codes are specified. 


6. Messages issued by modules must be defined, with text and identification numbers. 


7. Any design changes to high-level design since high-level design exit must be included in the 
current design specifications (high-level and low-level design materials must match). 


8. All design documents submitted for inspection should conform to local or project 
standards. 


Figure G7. Detailed design inspection exit criteria 
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FUNCTION PAXX-12(#1) DATE 5/4/77 


Error 
Description 


| Glassification 


M/W/E Min/Maj 


PAXX PCB address parameters in wrong sequence. 


1/O areas should be declared before the structures 
based on them. 


HEADSH(5) should contain CENO2 to match 
LINED; which should also refer to CENO2 
instead of CENO1. 


Security lines should have ASA char. of ‘c’ 


Blocks of DCL’s should be separated by comments. 


Base PICCONV on CHARCONV. 


TRAIL(6) is written to the wrong file. 


Comments on 1st occurrences of CALL’s. 
Message in case A should read “.... will not. ..”. 


PAXX201 | — — — 
| TP Ww Min 
PAXX302 | DA Ww 
LO WwW 


PAXX202 | LO W 
MN W 
DA WwW 


Figure G8. Sample error list 


Message does not refer to correct card type, when 
CO7 card is not present. 


A recycle # of ‘OX’ is treated incorrectly as valid. 
Should be included in test plan. 


By-name assignment into KEYN will not always 
work for PIC fields. 


‘RO3’ should be qualified by FPAOB not A. 


Positioning wrong when SUBST Ringing into 
MSG (111) 
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